Add a `.sqlite` database to be used to store rollout metatdata (and
later logs)
This PR is phase 1:
* Add the database and the required infrastructure
* Add a backfill of the database
* Persist the newly created rollout both in files and in the DB
* When we need to get metadata or a rollout, consider the `JSONL` as the
source of truth but compare the results with the DB and show any errors
This add a new crate, `codex-network-proxy`, a local network proxy
service used by Codex to enforce fine-grained network policy (domain
allow/deny) and to surface blocked network events for interactive
approvals.
- New crate: `codex-rs/network-proxy/` (`codex-network-proxy` binary +
library)
- Core capabilities:
- HTTP proxy support (including CONNECT tunneling)
- SOCKS5 proxy support (in the later PR)
- policy evaluation (allowed/denied domain lists; denylist wins;
wildcard support)
- small admin API for polling/reload/mode changes
- optional MITM support for HTTPS CONNECT to enforce “limited mode”
method restrictions (later PR)
Will follow up integration with codex in subsequent PRs.
## Testing
- `cd codex-rs && cargo build -p codex-network-proxy`
- `cd codex-rs && cargo run -p codex-network-proxy -- proxy`
Bumps [actions/cache](https://github.com/actions/cache) from 4 to 5.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/actions/cache/releases">actions/cache's
releases</a>.</em></p>
<blockquote>
<h2>v5.0.0</h2>
<blockquote>
<p>[!IMPORTANT]
<strong><code>actions/cache@v5</code> runs on the Node.js 24 runtime and
requires a minimum Actions Runner version of
<code>2.327.1</code>.</strong></p>
<p>If you are using self-hosted runners, ensure they are updated before
upgrading.</p>
</blockquote>
<hr />
<h2>What's Changed</h2>
<ul>
<li>Upgrade to use node24 by <a
href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1630">actions/cache#1630</a></li>
<li>Prepare v5.0.0 release by <a
href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1684">actions/cache#1684</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/cache/compare/v4.3.0...v5.0.0">https://github.com/actions/cache/compare/v4.3.0...v5.0.0</a></p>
<h2>v4.3.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Add note on runner versions by <a
href="https://github.com/GhadimiR"><code>@GhadimiR</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1642">actions/cache#1642</a></li>
<li>Prepare <code>v4.3.0</code> release by <a
href="https://github.com/Link"><code>@Link</code></a>- in <a
href="https://redirect.github.com/actions/cache/pull/1655">actions/cache#1655</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/GhadimiR"><code>@GhadimiR</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/cache/pull/1642">actions/cache#1642</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/cache/compare/v4...v4.3.0">https://github.com/actions/cache/compare/v4...v4.3.0</a></p>
<h2>v4.2.4</h2>
<h2>What's Changed</h2>
<ul>
<li>Update README.md by <a
href="https://github.com/nebuk89"><code>@nebuk89</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1620">actions/cache#1620</a></li>
<li>Upgrade <code>@actions/cache</code> to <code>4.0.5</code> and move
<code>@protobuf-ts/plugin</code> to dev depdencies by <a
href="https://github.com/Link"><code>@Link</code></a>- in <a
href="https://redirect.github.com/actions/cache/pull/1634">actions/cache#1634</a></li>
<li>Prepare release <code>4.2.4</code> by <a
href="https://github.com/Link"><code>@Link</code></a>- in <a
href="https://redirect.github.com/actions/cache/pull/1636">actions/cache#1636</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/nebuk89"><code>@nebuk89</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/cache/pull/1620">actions/cache#1620</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/cache/compare/v4...v4.2.4">https://github.com/actions/cache/compare/v4...v4.2.4</a></p>
<h2>v4.2.3</h2>
<h2>What's Changed</h2>
<ul>
<li>Update to use <code>@actions/cache</code> 4.0.3 package &
prepare for new release by <a
href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1577">actions/cache#1577</a>
(SAS tokens for cache entries are now masked in debug logs)</li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/cache/pull/1577">actions/cache#1577</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/cache/compare/v4.2.2...v4.2.3">https://github.com/actions/cache/compare/v4.2.2...v4.2.3</a></p>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/actions/cache/blob/main/RELEASES.md">actions/cache's
changelog</a>.</em></p>
<blockquote>
<h1>Releases</h1>
<h2>Changelog</h2>
<h3>5.0.1</h3>
<ul>
<li>Update <code>@azure/storage-blob</code> to <code>^12.29.1</code> via
<code>@actions/cache@5.0.1</code> <a
href="https://redirect.github.com/actions/cache/pull/1685">#1685</a></li>
</ul>
<h3>5.0.0</h3>
<blockquote>
<p>[!IMPORTANT]
<code>actions/cache@v5</code> runs on the Node.js 24 runtime and
requires a minimum Actions Runner version of <code>2.327.1</code>.
If you are using self-hosted runners, ensure they are updated before
upgrading.</p>
</blockquote>
<h3>4.3.0</h3>
<ul>
<li>Bump <code>@actions/cache</code> to <a
href="https://redirect.github.com/actions/toolkit/pull/2132">v4.1.0</a></li>
</ul>
<h3>4.2.4</h3>
<ul>
<li>Bump <code>@actions/cache</code> to v4.0.5</li>
</ul>
<h3>4.2.3</h3>
<ul>
<li>Bump <code>@actions/cache</code> to v4.0.3 (obfuscates SAS token in
debug logs for cache entries)</li>
</ul>
<h3>4.2.2</h3>
<ul>
<li>Bump <code>@actions/cache</code> to v4.0.2</li>
</ul>
<h3>4.2.1</h3>
<ul>
<li>Bump <code>@actions/cache</code> to v4.0.1</li>
</ul>
<h3>4.2.0</h3>
<p>TLDR; The cache backend service has been rewritten from the ground up
for improved performance and reliability. <a
href="https://github.com/actions/cache">actions/cache</a> now integrates
with the new cache service (v2) APIs.</p>
<p>The new service will gradually roll out as of <strong>February 1st,
2025</strong>. The legacy service will also be sunset on the same date.
Changes in these release are <strong>fully backward
compatible</strong>.</p>
<p><strong>We are deprecating some versions of this action</strong>. We
recommend upgrading to version <code>v4</code> or <code>v3</code> as
soon as possible before <strong>February 1st, 2025.</strong> (Upgrade
instructions below).</p>
<p>If you are using pinned SHAs, please use the SHAs of versions
<code>v4.2.0</code> or <code>v3.4.0</code></p>
<p>If you do not upgrade, all workflow runs using any of the deprecated
<a href="https://github.com/actions/cache">actions/cache</a> will
fail.</p>
<p>Upgrading to the recommended versions will not break your
workflows.</p>
<h3>4.1.2</h3>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="9255dc7a25"><code>9255dc7</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/cache/issues/1686">#1686</a>
from actions/cache-v5.0.1-release</li>
<li><a
href="8ff5423e8b"><code>8ff5423</code></a>
chore: release v5.0.1</li>
<li><a
href="9233019a15"><code>9233019</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/cache/issues/1685">#1685</a>
from salmanmkc/node24-storage-blob-fix</li>
<li><a
href="b975f2bb84"><code>b975f2b</code></a>
fix: add peer property to package-lock.json for dependencies</li>
<li><a
href="d0a0e18134"><code>d0a0e18</code></a>
fix: update license files for <code>@actions/cache</code>,
fast-xml-parser, and strnum</li>
<li><a
href="74de208dcf"><code>74de208</code></a>
fix: update <code>@actions/cache</code> to ^5.0.1 for Node.js 24
punycode fix</li>
<li><a
href="ac7f1152ea"><code>ac7f115</code></a>
peer</li>
<li><a
href="b0f846b50b"><code>b0f846b</code></a>
fix: update <code>@actions/cache</code> with storage-blob fix for
Node.js 24 punycode depr...</li>
<li><a
href="a783357455"><code>a783357</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/cache/issues/1684">#1684</a>
from actions/prepare-cache-v5-release</li>
<li><a
href="3bb0d78750"><code>3bb0d78</code></a>
docs: highlight v5 runner requirement in releases</li>
<li>Additional commits viewable in <a
href="https://github.com/actions/cache/compare/v4...v5">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
When I originally introduced `accept_elicitation_for_prompt_rule()` in
https://github.com/openai/codex/pull/7617, it worked for me locally
because I had run `codex-rs/exec-server/tests/suite/bash` once myself,
which had the side-effect of installing the corresponding DotSlash
artifact.
In CI, I added explicit logic to do this as part of
`.github/workflows/rust-ci.yml`, which meant the test also passed in CI,
but this logic should have been done as part of the test so that it
would work locally for devs who had not installed the DotSlash artifact
for `codex-rs/exec-server/tests/suite/bash` before. This PR updates the
test to do this (and deletes the setup logic from `rust-ci.yml`),
creating a new `DOTSLASH_CACHE` in a temp directory so that this is
handled independently for each test.
While here, also added a check to ensure that the `codex` binary has
been built prior to running the test, as we have to ensure it is
symlinked as `codex-linux-sandbox` on Linux in order for the integration
test to work on that platform.
This PR introduces integration tests that run
[codex-shell-tool-mcp](https://www.npmjs.com/package/@openai/codex-shell-tool-mcp)
as a user would. Note that this requires running our fork of Bash, so we
introduce a [DotSlash](https://dotslash-cli.com/) file for `bash` so
that we can run the integration tests on multiple platforms without
having to check the binaries into the repository. (As noted in the
DotSlash file, it is slightly more heavyweight than necessary, which may
be worth addressing as disk space in CI is limited:
https://github.com/openai/codex/pull/7678.)
To start, this PR adds two tests:
- `list_tools()` makes the `list_tools` request to the MCP server and
verifies we get the expected response
- `accept_elicitation_for_prompt_rule()` defines a `prefix_rule()` with
`decision="prompt"` and verifies the elicitation flow works as expected
Though the `accept_elicitation_for_prompt_rule()` test **only works on
Linux**, as this PR reveals that there are currently issues when running
the Bash fork in a read-only sandbox on Linux. This will have to be
fixed in a follow-up PR.
Incidentally, getting this test run to correctly on macOS also requires
a recent fix we made to `brew` that hasn't hit a mainline release yet,
so getting CI green in this PR required
https://github.com/openai/codex/pull/7680.
When I put up https://github.com/openai/codex/pull/7617 for review,
initially I started seeing failures on the `ubuntu-24.04` runner used
for Rust test runs for the `x86_64-unknown-linux-gnu` architecture. Chat
suggested a number of things that could be removed to save space, which
seems to help.
Bumps
[taiki-e/install-action](https://github.com/taiki-e/install-action) from
2.60.0 to 2.62.49.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/taiki-e/install-action/releases">taiki-e/install-action's
releases</a>.</em></p>
<blockquote>
<h2>2.62.49</h2>
<ul>
<li>
<p>Update <code>cargo-binstall@latest</code> to 1.15.11.</p>
</li>
<li>
<p>Update <code>cargo-auditable@latest</code> to 0.7.2.</p>
</li>
<li>
<p>Update <code>vacuum@latest</code> to 0.20.2.</p>
</li>
</ul>
<h2>2.62.48</h2>
<ul>
<li>
<p>Update <code>mise@latest</code> to 2025.11.3.</p>
</li>
<li>
<p>Update <code>cargo-audit@latest</code> to 0.22.0.</p>
</li>
<li>
<p>Update <code>vacuum@latest</code> to 0.20.1.</p>
</li>
<li>
<p>Update <code>uv@latest</code> to 0.9.8.</p>
</li>
<li>
<p>Update <code>cargo-udeps@latest</code> to 0.1.60.</p>
</li>
<li>
<p>Update <code>zizmor@latest</code> to 1.16.3.</p>
</li>
</ul>
<h2>2.62.47</h2>
<ul>
<li>
<p>Update <code>vacuum@latest</code> to 0.20.0.</p>
</li>
<li>
<p>Update <code>cargo-nextest@latest</code> to 0.9.111.</p>
</li>
<li>
<p>Update <code>cargo-shear@latest</code> to 1.6.2.</p>
</li>
</ul>
<h2>2.62.46</h2>
<ul>
<li>
<p>Update <code>vacuum@latest</code> to 0.19.5.</p>
</li>
<li>
<p>Update <code>syft@latest</code> to 1.37.0.</p>
</li>
<li>
<p>Update <code>mise@latest</code> to 2025.11.2.</p>
</li>
<li>
<p>Update <code>knope@latest</code> to 0.21.5.</p>
</li>
</ul>
<h2>2.62.45</h2>
<ul>
<li>
<p>Update <code>zizmor@latest</code> to 1.16.2.</p>
</li>
<li>
<p>Update <code>cargo-binstall@latest</code> to 1.15.10.</p>
</li>
<li>
<p>Update <code>ubi@latest</code> to 0.8.4.</p>
</li>
<li>
<p>Update <code>mise@latest</code> to 2025.11.1.</p>
</li>
<li>
<p>Update <code>cargo-semver-checks@latest</code> to 0.45.0.</p>
</li>
</ul>
<h2>2.62.44</h2>
<ul>
<li>Update <code>mise@latest</code> to 2025.11.0.</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md">taiki-e/install-action's
changelog</a>.</em></p>
<blockquote>
<h1>Changelog</h1>
<p>All notable changes to this project will be documented in this
file.</p>
<p>This project adheres to <a href="https://semver.org">Semantic
Versioning</a>.</p>
<!-- raw HTML omitted -->
<h2>[Unreleased]</h2>
<h2>[2.62.49] - 2025-11-09</h2>
<ul>
<li>
<p>Update <code>cargo-binstall@latest</code> to 1.15.11.</p>
</li>
<li>
<p>Update <code>cargo-auditable@latest</code> to 0.7.2.</p>
</li>
<li>
<p>Update <code>vacuum@latest</code> to 0.20.2.</p>
</li>
</ul>
<h2>[2.62.48] - 2025-11-08</h2>
<ul>
<li>
<p>Update <code>mise@latest</code> to 2025.11.3.</p>
</li>
<li>
<p>Update <code>cargo-audit@latest</code> to 0.22.0.</p>
</li>
<li>
<p>Update <code>vacuum@latest</code> to 0.20.1.</p>
</li>
<li>
<p>Update <code>uv@latest</code> to 0.9.8.</p>
</li>
<li>
<p>Update <code>cargo-udeps@latest</code> to 0.1.60.</p>
</li>
<li>
<p>Update <code>zizmor@latest</code> to 1.16.3.</p>
</li>
</ul>
<h2>[2.62.47] - 2025-11-05</h2>
<ul>
<li>
<p>Update <code>vacuum@latest</code> to 0.20.0.</p>
</li>
<li>
<p>Update <code>cargo-nextest@latest</code> to 0.9.111.</p>
</li>
<li>
<p>Update <code>cargo-shear@latest</code> to 1.6.2.</p>
</li>
</ul>
<h2>[2.62.46] - 2025-11-04</h2>
<ul>
<li>
<p>Update <code>vacuum@latest</code> to 0.19.5.</p>
</li>
<li>
<p>Update <code>syft@latest</code> to 1.37.0.</p>
</li>
<li>
<p>Update <code>mise@latest</code> to 2025.11.2.</p>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="44c6d64aa6"><code>44c6d64</code></a>
Release 2.62.49</li>
<li><a
href="3a701df4c2"><code>3a701df</code></a>
Update <code>cargo-binstall@latest</code> to 1.15.11</li>
<li><a
href="4242e04eb8"><code>4242e04</code></a>
Update <code>cargo-auditable@latest</code> to 0.7.2</li>
<li><a
href="3df5533ef8"><code>3df5533</code></a>
Update <code>vacuum@latest</code> to 0.20.2</li>
<li><a
href="e797ba6a25"><code>e797ba6</code></a>
Release 2.62.48</li>
<li><a
href="bcf91e02ac"><code>bcf91e0</code></a>
Update <code>mise@latest</code> to 2025.11.3</li>
<li><a
href="e78113b60c"><code>e78113b</code></a>
Update <code>cargo-audit@latest</code> to 0.22.0</li>
<li><a
href="0ef486444e"><code>0ef4864</code></a>
Update <code>vacuum@latest</code> to 0.20.1</li>
<li><a
href="5eda7b1985"><code>5eda7b1</code></a>
Update <code>uv@latest</code> to 0.9.8</li>
<li><a
href="3853a413e6"><code>3853a41</code></a>
Update <code>cargo-udeps@latest</code> to 0.1.60</li>
<li>Additional commits viewable in <a
href="0c5db7f7f8...44c6d64aa6">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
## Summary
musl 1.2.5 includes [several fixes to DNS over
TCP](https://www.openwall.com/lists/musl/2024/03/01/2), which appears to
be the root cause of #6116.
This approach is a bit janky, but according to codex:
> On the Ubuntu 24.04 runners we use, apt-cache policy musl-tools shows
only the distro build (1.2.4-2ubuntu2)"
We should build with this version and confirm.
## Testing
- [ ] TODO: test and see if this fixes Azure issues
As a follow-up to https://github.com/openai/codex/pull/3439, this adds a
CI job to ensure the codegen script has to be updated in order to change
`codex-rs/mcp-types/src/lib.rs`.
I started looking at https://nexte.st/ because I was interested in a
test harness that lets a test dynamically declare itself "skipped,"
which would be a nice alternative to this pattern:
4c46490e53/codex-rs/core/tests/suite/cli_stream.rs (L22-L27)
ChatGPT pointed me at https://nexte.st/, which also claims to be "up to
3x as fast as cargo test." Locally, in `codex-rs`, I see
- `cargo nextest run` finishes in 19s
- `cargo test` finishes in 37s
Though looking at CI, the wins are quite as big, presumably because my
laptop has more cores than our GitHub runners (which is a separate
issue...). Comparing the [CI jobs from this
PR](https://github.com/openai/codex/actions/runs/17561325162/job/49878216246?pr=3323)
with that of a [recent open
PR](https://github.com/openai/codex/actions/runs/17561066581/job/49877342753?pr=3321):
| | `cargo test` | `cargo nextest` |
| ----------------------------------------------- | ------------ |
--------------- |
| `macos-14 - aarch64-apple-darwin` | 2m16s | 1m51s |
| `macos-14 - aarch64-apple-darwin` | 5m04s | 3m44s |
| `ubuntu-24.04 - x86_64-unknown-linux-musl` | 2m02s | 1m56s |
| `ubuntu-24.04-arm - aarch64-unknown-linux-musl` | 2m01s | 1m35s |
| `windows-latest - x86_64-pc-windows-msvc` | 3m07s | 2m53s |
| `windows-11-arm - aarch64-pc-windows-msvc` | 3m10s | 2m45s |
I thought that, to start, we would only make this change in CI before
declaring it the "official" way for the team to run the test suite.
Though unfortunately, I do not believe that `cargo nextest` _actually_
supports a dynamic skip feature, so I guess I'll have to keep looking?
Some related discussions:
- https://internals.rust-lang.org/t/pre-rfc-skippable-tests/14611
- https://internals.rust-lang.org/t/skippable-tests/21260
This is in support of https://github.com/openai/codex/issues/2979.
Once we have a release out, we can update the npm module and the VS Code
extension to take advantage of this.
Today we had a breakage in the release build that went unnoticed by CI.
Here is what happened:
- https://github.com/openai/codex/pull/2242 originally added some logic
to do release builds to prevent this from happening
- https://github.com/openai/codex/pull/2276 undid that change to try to
speed things up by removing the step to build all the individual crates
in release mode, assuming the `cargo check` call was sufficient
coverage, which it would have been, had it specified `--profile`
This PR adds `--profile` to the `cargo check` step so we should get the
desired coverage from our build matrix.
Indeed, enabling this in our CI uncovered a warning that is only present
in release mode that was going unnoticed.
Codex created this PR from the following prompt:
> upgrade this entire repo to Rust 1.89. Note that this requires
updating codex-rs/rust-toolchain.toml as well as the workflows in
.github/. Make sure that things are "clippy clean" as this change will
likely uncover new Clippy errors. `just fmt` and `cargo clippy --tests`
are sufficient to check for correctness
Note this modifies a lot of lines because it folds nested `if`
statements using `&&`.
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/2465).
* #2467
* __->__ #2465
The `ubuntu-24.04 - x86_64-unknown-linux-gnu` build is failing with `No
space left on device` on #2465, so let's get this in first, which should
help.
Note that `cargo check` should be faster and use less disk than `cargo
build` because it does not write out the object files.
It turns out that https://github.com/openai/codex/pull/2324 did not
quite work as intended. Chat's new idea is to have this catch-all "CI
results" job and update our branch protection rules to require this
instead.
Our existing path filters for `rust-ci.yml`:
235987843c/.github/workflows/rust-ci.yml (L1-L11)
made it so that PRs that touch only `README.md` would not trigger those
builds, which is a problem because our branch protection rules are set
as follows:
<img width="1569" height="1883" alt="Screenshot 2025-08-14 at 4 45
59 PM"
src="https://github.com/user-attachments/assets/5a61f8cc-cdaf-4341-abda-7faa7b46dbd4"
/>
With the existing setup, a change to `README.md` would get stuck in
limbo because not all the CI jobs required to merge would get run. It
turns out that we need to "run" all the jobs, but make them no-ops when
the `codex-rs` and `.github` folders are untouched to get the best of
both worlds.
I asked chat how to fix this, as we want CI to be fast for
documentation-only changes. It had two suggestions:
- Use https://github.com/dorny/paths-filter or some other third-party
action.
- Write an inline Bash script to avoid a third-party dependency.
This PR takes the latter approach so that we are clear about what we're
running in CI.
I put this PR together because I noticed I have to wait quite a bit
longer on my PRs since we added
https://github.com/openai/codex/pull/2242 to catch more build issues.
I think we should think about reigning in our use of create features,
but this should be good enough to speed things up for now.
Users were running into issues with glibc mismatches on arm64 linux. In
the past, we did not provide a musl build for arm64 Linux because we had
trouble getting the openssl dependency to build correctly. Though today
I just tried the same trick in `Cargo.toml` that we were doing for
`x86_64-unknown-linux-musl` (using `openssl-sys` with `features =
["vendored"]`), so I'm not sure what problem we had in the past the
builds "just worked" today!
Though one tweak that did have to be made is that the integration tests
for Seccomp/Landlock empirically require longer timeouts on arm64 linux,
or at least on the `ubuntu-24.04-arm` GitHub Runner. As such, we change
the timeouts for arm64 in `codex-rs/linux-sandbox/tests/landlock.rs`.
Though in solving this problem, I decided I needed a turnkey solution
for testing the Linux build(s) from my Mac laptop, so this PR introduces
`.devcontainer/Dockerfile` and `.devcontainer/devcontainer.json` to
facilitate this. Detailed instructions are in `.devcontainer/README.md`.
We will update `dotslash-config.json` and other release-related scripts
in a follow-up PR.
To date, when handling `shell` and `local_shell` tool calls, we were
spawning new processes using the environment inherited from the Codex
process itself. This means that the sensitive `OPENAI_API_KEY` that
Codex needs to talk to OpenAI models was made available to everything
run by `shell` and `local_shell`. While there are cases where that might
be useful, it does not seem like a good default.
This PR introduces a complex `shell_environment_policy` config option to
control the `env` used with these tool calls. It is inevitably a bit
complex so that it is possible to override individual components of the
policy so without having to restate the entire thing.
Details are in the updated `README.md` in this PR, but here is the
relevant bit that explains the individual fields of
`shell_environment_policy`:
| Field | Type | Default | Description |
| ------------------------- | -------------------------- | ------- |
-----------------------------------------------------------------------------------------------------------------------------------------------
|
| `inherit` | string | `core` | Starting template for the
environment:<br>`core` (`HOME`, `PATH`, `USER`, …), `all` (clone full
parent env), or `none` (start empty). |
| `ignore_default_excludes` | boolean | `false` | When `false`, Codex
removes any var whose **name** contains `KEY`, `SECRET`, or `TOKEN`
(case-insensitive) before other rules run. |
| `exclude` | array<string> | `[]` | Case-insensitive glob
patterns to drop after the default filter.<br>Examples: `"AWS_*"`,
`"AZURE_*"`. |
| `set` | table<string,string> | `{}` | Explicit key/value
overrides or additions – always win over inherited values. |
| `include_only` | array<string> | `[]` | If non-empty, a
whitelist of patterns; only variables that match _one_ pattern survive
the final step. (Generally used with `inherit = "all"`.) |
In particular, note that the default is `inherit = "core"`, so:
* if you have extra env variables that you want to inherit from the
parent process, use `inherit = "all"` and then specify `include_only`
* if you have extra env variables where you want to hardcode the values,
the default `inherit = "core"` will work fine, but then you need to
specify `set`
This configuration is not battle-tested, so we will probably still have
to play with it a bit. `core/src/exec_env.rs` has the critical business
logic as well as unit tests.
Though if nothing else, previous to this change:
```
$ cargo run --bin codex -- debug seatbelt -- printenv OPENAI_API_KEY
# ...prints OPENAI_API_KEY...
```
But after this change it does not print anything (as desired).
One final thing to call out about this PR is that the
`configure_command!` macro we use in `core/src/exec.rs` has to do some
complex logic with respect to how it builds up the `env` for the process
being spawned under Landlock/seccomp. Specifically, doing
`cmd.env_clear()` followed by `cmd.envs(&$env_map)` (which is arguably
the most intuitive way to do it) caused the Landlock unit tests to fail
because the processes spawned by the unit tests started failing in
unexpected ways! If we forgo `env_clear()` in favor of updating env vars
one at a time, the tests still pass. The comment in the code talks about
this a bit, and while I would like to investigate this more, I need to
move on for the moment, but I do plan to come back to it to fully
understand what is going on. For example, this suggests that we might
not be able to spawn a C program that calls `env_clear()`, which would
be...weird. We may still have to fiddle with our Landlock config if that
is the case.
Moving to Rust 1.87 introduced a clippy warning that
`SendError<AppEvent>` was too large.
In practice, the only thing we ever did when we got this error was log
it (if the mspc channel is closed, then the app is likely shutting down
or something, so there's not much to do...), so this finally motivated
me to introduce `AppEventSender`, which wraps
`std::sync::mpsc::Sender<AppEvent>` with a `send()` method that invokes
`send()` on the underlying `Sender` and logs an `Err` if it gets one.
This greatly simplifies the code, as many functions that previously
returned `Result<(), SendError<AppEvent>>` now return `()`, so we don't
have to propagate an `Err` all over the place that we don't really
handle, anyway.
This also makes it so we can upgrade to Rust 1.87 in CI.
Previously, our GitHub actions specified the Rust toolchain as
`dtolnay/rust-toolchain@stable`, which meant the version could change
out from under us. In this case, the move from 1.86 to 1.87 introduced
new clippy warnings, causing build failures.
Because it will take a little time to fix all the new clippy warnings,
this PR pins things to 1.86 for now to unbreak the build.
It also replaces `io::Error::new(io::ErrorKind::Other)` with
`io::Error::other()` in preparation for 1.87.
I installed the GitHub Actions extension for VS Code and it started
giving me lint warnings about this line:
a9adb4175c/.github/workflows/rust-ci.yml (L99)
Using an env var to track the state of individual steps was not great,
so I did some research about GitHub actions, which led to the discovery
of combining `continue-on-error: true` with `if .. steps.STEP.outcome ==
'failure'...`.
Apparently there is also a `failure()` macro that is supposed to make
this simpler, but I saw a number of complains online about it not
working as expected. Checking `outcome` seems maybe more reliable at the
cost of being slightly more verbose.
https://github.com/openai/codex/pull/855 added the clippy warning to
disallow `unwrap()`, but apparently we were not verifying that tests
were "clippy clean" in CI, so I ended up with a lot of local errors in
VS Code.
This turns on the check in CI and fixes the offenders.
I started this PR because I wanted to share the `format_duration()`
utility function in `codex-rs/exec/src/event_processor.rs` with the TUI.
The question was: where to put it?
`core` should have as few dependencies as possible, so moving it there
would introduce a dependency on `chrono`, which seemed undesirable.
`core` already had this `cli` feature to deal with a similar situation
around sharing common utility functions, so I decided to:
* make `core` feature-free
* introduce `common`
* `common` can have as many "special interest" features as it needs,
each of which can declare their own deps
* the first two features of common are `cli` and `elapsed`
In practice, this meant updating a number of `Cargo.toml` files,
replacing this line:
```toml
codex-core = { path = "../core", features = ["cli"] }
```
with these:
```toml
codex-core = { path = "../core" }
codex-common = { path = "../common", features = ["cli"] }
```
Moving `format_duration()` into its own file gave it some "breathing
room" to add a unit test, so I had Codex generate some tests and new
support for durations over 1 minute.
I discovered that `cargo build` worked for the entire workspace, but not
for the `mcp-client` or `core` crates.
* `mcp-client` failed to build because it underspecified the set of
features it needed from `tokio`.
* `core` failed to build because it was using a "feature" of its own
crate in the default, no-feature version.
This PR fixes the builds and adds a check in CI to defend against this
sort of thing going forward.
