e61bae12e3
This PR introduces a `codex-utils-cargo-bin` utility crate that wraps/replaces our use of `assert_cmd::Command` and `escargot::CargoBuild`. As you can infer from the introduction of `buck_project_root()` in this PR, I am attempting to make it possible to build Codex under [Buck2](https://buck2.build) as well as `cargo`. With Buck2, I hope to achieve faster incremental local builds (largely due to Buck2's [dice](https://buck2.build/docs/insights_and_knowledge/modern_dice/) build strategy, as well as benefits from its local build daemon) as well as faster CI builds if we invest in remote execution and caching. See https://buck2.build/docs/getting_started/what_is_buck2/#why-use-buck2-key-advantages for more details about the performance advantages of Buck2. Buck2 enforces stronger requirements in terms of build and test isolation. It discourages assumptions about absolute paths (which is key to enabling remote execution). Because the `CARGO_BIN_EXE_*` environment variables that Cargo provides are absolute paths (which `assert_cmd::Command` reads), this is a problem for Buck2, which is why we need this `codex-utils-cargo-bin` utility. My WIP-Buck2 setup sets the `CARGO_BIN_EXE_*` environment variables passed to a `rust_test()` build rule as relative paths. `codex-utils-cargo-bin` will resolve these values to absolute paths, when necessary. --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/openai/codex/pull/8496). * #8498 * __->__ #8496 |
||
|---|---|---|
| .. | ||
| src | ||
| tests | ||
| Cargo.toml | ||
| README.md | ||
codex-exec-server
This crate contains the code for two executables:
codex-exec-mcp-serveris an MCP server that provides a tool namedshellthat runs a shell command inside a sandboxed instance of Bash. Every resultingexecve(2)call made within Bash is intercepted and run via the executable defined by theBASH_EXEC_WRAPPERenvironment variable within the Bash process. In practice,BASH_EXEC_WRAPPERis set tocodex-execve-wrapper.codex-execve-wrapperis the executable that takes the arguments to theexecve(2)call and "escalates" it to the MCP server via a shared file descriptor (specified by theCODEX_ESCALATE_SOCKETenvironment variable) for consideration. Based on the Codex.rules, the MCP server replies with one of:Run:codex-execve-wrappershould invokeexecve(2)on itself to run the original command within BashEscalate: forward the file descriptors of the current process to the MCP server so the command can be run faithfully outside the sandbox. Because the MCP server will have the original FDs forstdoutandstderr, it can write those directly. When the process completes, the MCP server forwards the exit code tocodex-execve-wrapperso that it exits in a consistent manner.Deny: the MCP server has declared the proposed command to be "forbidden," socodex-execve-wrapperwill print an error tostderrand exit with1.
Patched Bash
We carry a small patch to execute_cmd.c (see patches/bash-exec-wrapper.patch) that adds support for BASH_EXEC_WRAPPER. The original commit message is “add support for BASH_EXEC_WRAPPER” and the patch applies cleanly to a8a1c2fac029404d3f42cd39f5a20f24b6e4fe4b from https://github.com/bminor/bash. To rebuild manually:
git clone https://github.com/bminor/bash
git checkout a8a1c2fac029404d3f42cd39f5a20f24b6e4fe4b
git apply /path/to/patches/bash-exec-wrapper.patch
./configure --without-bash-malloc
make -j"$(nproc)"
Release workflow
.github/workflows/shell-tool-mcp.yml builds the Rust binaries, compiles the patched Bash variants, assembles the vendor/ tree, and creates codex-shell-tool-mcp-npm-<version>.tgz for inclusion in the Rust GitHub Release. When the version is a stable or alpha tag, the workflow also publishes the tarball to npm using OIDC. The workflow is invoked from rust-release.yml so the package ships alongside other Codex artifacts.