core-agent-ide/codex-rs/codex-client/tests/ca_env.rs

146 lines
5 KiB
Rust
Raw Permalink Normal View History

login: add custom CA support for login flows (#14178) ## Stacked PRs This work is split across three stacked PRs: - #14178: add custom CA support for browser and device-code login flows, docs, and hermetic subprocess tests - #14239: broaden the shared custom CA path from login to other outbound `reqwest` clients across Codex - #14240: extend that shared custom CA handling to secure websocket TLS so websocket connections honor the same CA env vars Review order: #14178, then #14239, then #14240. Supersedes #6864. Thanks to @3axap4eHko for the original implementation and investigation here. Although this version rearranges the code and history significantly, the majority of the credit for this work belongs to them. ## Problem Login flows need to work in enterprise environments where outbound TLS is intercepted by an internal proxy or gateway. In those setups, system root certificates alone are often insufficient to validate the OAuth and device-code endpoints used during login. The change adds a login-specific custom CA loading path, but the important contracts around env precedence, PEM compatibility, test boundaries, and probe-only workarounds need to be explicit so reviewers can understand what behavior is intentional. For users and operators, the behavior is simple: if login needs to trust a custom root CA, set `CODEX_CA_CERTIFICATE` to a PEM file containing one or more certificates. If that variable is unset, login falls back to `SSL_CERT_FILE`. If neither is set, login uses system roots. Invalid or empty PEM files now fail with an error that points back to those environment variables and explains how to recover. ## What This Delivers Users can now make Codex login work behind enterprise TLS interception by pointing `CODEX_CA_CERTIFICATE` at a PEM bundle containing the relevant root certificates. If that variable is unset, login falls back to `SSL_CERT_FILE`, then to system roots. This PR applies that behavior to both browser-based and device-code login flows. It also makes login tolerant of the PEM shapes operators actually have in hand: multi-certificate bundles, OpenSSL `TRUSTED CERTIFICATE` labels, and bundles that include well-formed CRLs. ## Mental model `codex-login` is the place where the login flows construct ad hoc outbound HTTP clients. That makes it the right boundary for a narrow CA policy: look for `CODEX_CA_CERTIFICATE`, fall back to `SSL_CERT_FILE`, load every parseable certificate block in that bundle into a `reqwest::Client`, and fail early with a clear user-facing error if the bundle is unreadable or malformed. The implementation is intentionally pragmatic about PEM input shape. It accepts ordinary certificate bundles, multi-certificate bundles, OpenSSL `TRUSTED CERTIFICATE` labels, and bundles that also contain CRLs. It does not validate a certificate chain or prove a handshake; it only constructs the root store used by login. ## Non-goals This change does not introduce a general-purpose transport abstraction for the rest of the product. It does not validate whether the provided bundle forms a real chain, and it does not add handshake-level integration tests against a live TLS server. It also does not change login state management or OAuth semantics beyond ensuring the existing flows share the same CA-loading rules. ## Tradeoffs The main tradeoff is keeping this logic scoped to login-specific client construction rather than lifting it into a broader shared HTTP layer. That keeps the review surface smaller, but it also means future login-adjacent code must continue to use `build_login_http_client()` or it can silently bypass enterprise CA overrides. The `TRUSTED CERTIFICATE` handling is also intentionally a local compatibility shim. The rustls ecosystem does not currently accept that PEM label upstream, so the code normalizes it locally and trims the OpenSSL `X509_AUX` trailer bytes down to the certificate DER that `reqwest` can consume. ## Architecture `custom_ca.rs` is now the single place that owns login CA behavior. It selects the CA file from the environment, reads it, normalizes PEM label shape where needed, iterates mixed PEM sections with `rustls-pki-types`, ignores CRLs, trims OpenSSL trust metadata when necessary, and returns either a configured `reqwest::Client` or a typed error. The browser login server and the device-code flow both call `build_login_http_client()`, so they share the same trust-store policy. Environment-sensitive tests run through the `login_ca_probe` helper binary because those tests must control process-wide env vars and cannot reliably build a real reqwest client in-process on macOS seatbelt runs. ## Observability The custom CA path logs which environment variable selected the bundle, which file path was loaded, how many certificates were accepted, when `TRUSTED CERTIFICATE` labels were normalized, when CRLs were ignored, and where client construction failed. Returned errors remain user-facing and include the relevant path, env var, and remediation hint. This gives enough signal for three audiences: - users can see why login failed and which env/file caused it - sysadmins can confirm which override actually won - developers can tell whether the failure happened during file read, PEM parsing, certificate registration, or final reqwest client construction ## Tests Pure unit tests stay limited to env precedence and empty-value handling. Real client construction lives in subprocess tests so the suite remains hermetic with respect to process env and macOS sandbox behavior. The subprocess tests verify: - `CODEX_CA_CERTIFICATE` precedence over `SSL_CERT_FILE` - fallback to `SSL_CERT_FILE` - single-certificate and multi-certificate bundles - malformed and empty-bundle errors - OpenSSL `TRUSTED CERTIFICATE` handling - CRL tolerance for well-formed CRL sections The named PEM fixtures under `login/tests/fixtures/` are shared by the tests so their purpose stays reviewable. --------- Co-authored-by: Ivan Zakharchanka <3axap4eHko@gmail.com> Co-authored-by: Codex <noreply@openai.com>
2026-03-12 17:14:54 -07:00
//! Subprocess coverage for custom CA behavior that must build a real reqwest client.
//!
client: extend custom CA handling across HTTPS and websocket clients (#14239) ## Stacked PRs This work is now effectively split across two steps: - #14178: add custom CA support for browser and device-code login flows, docs, and hermetic subprocess tests - #14239: extend that shared custom CA handling across Codex HTTPS clients and secure websocket TLS Note: #14240 was merged into this branch while it was stacked on top of this PR. This PR now subsumes that websocket follow-up and should be treated as the combined change. Builds on top of #14178. ## Problem Custom CA support landed first in the login path, but the real requirement is broader. Codex constructs outbound TLS clients in multiple places, and both HTTPS and secure websocket paths can fail behind enterprise TLS interception if they do not honor `CODEX_CA_CERTIFICATE` or `SSL_CERT_FILE` consistently. This PR broadens the shared custom-CA logic beyond login and applies the same policy to websocket TLS, so the enterprise-proxy story is no longer split between “HTTPS works” and “websockets still fail”. ## What This Delivers Custom CA support is no longer limited to login. Codex outbound HTTPS clients and secure websocket connections can now honor the same `CODEX_CA_CERTIFICATE` / `SSL_CERT_FILE` configuration, so enterprise proxy/intercept setups work more consistently end-to-end. For users and operators, nothing new needs to be configured beyond the same CA env vars introduced in #14178. The change is that more of Codex now respects them, including websocket-backed flows that were previously still using default trust roots. I also manually validated the proxy path locally with mitmproxy using: `CODEX_CA_CERTIFICATE=~/.mitmproxy/mitmproxy-ca-cert.pem HTTPS_PROXY=http://127.0.0.1:8080 just codex` with mitmproxy installed via `brew install mitmproxy` and configured as the macOS system proxy. ## Mental model `codex-client` is now the owner of shared custom-CA policy for outbound TLS client construction. Reqwest callers start from the builder configuration they already need, then pass that builder through `build_reqwest_client_with_custom_ca(...)`. Websocket callers ask the same module for a rustls client config when a custom CA bundle is configured. The env precedence is the same everywhere: - `CODEX_CA_CERTIFICATE` wins - otherwise fall back to `SSL_CERT_FILE` - otherwise use system roots The helper is intentionally narrow. It loads every usable certificate from the configured PEM bundle into the appropriate root store and returns either a configured transport or a typed error that explains what went wrong. ## Non-goals This does not add handshake-level integration tests against a live TLS endpoint. It does not validate that the configured bundle forms a meaningful certificate chain. It also does not try to force every transport in the repo through one abstraction; it extends the shared CA policy across the reqwest and websocket paths that actually needed it. ## Tradeoffs The main tradeoff is centralizing CA behavior in `codex-client` while still leaving adoption up to call sites. That keeps the implementation additive and reviewable, but it means the rule "outbound Codex TLS that should honor enterprise roots must use the shared helper" is still partly enforced socially rather than by types. For websockets, the shared helper only builds an explicit rustls config when a custom CA bundle is configured. When no override env var is set, websocket callers still use their ordinary default connector path. ## Architecture `codex-client::custom_ca` now owns CA bundle selection, PEM normalization, mixed-section parsing, certificate extraction, typed CA-loading errors, and optional rustls client-config construction for websocket TLS. The affected consumers now call into that shared helper directly rather than carrying login-local CA behavior: - backend-client - cloud-tasks - RMCP client paths that use `reqwest` - TUI voice HTTP paths - `codex-core` default reqwest client construction - `codex-api` websocket clients for both responses and realtime websocket connections The subprocess CA probe, env-sensitive integration tests, and shared PEM fixtures also live in `codex-client`, which is now the actual owner of the behavior they exercise. ## Observability The shared CA path logs: - which environment variable selected the bundle - which path was loaded - how many certificates were accepted - when `TRUSTED CERTIFICATE` labels were normalized - when CRLs were ignored - where client construction failed Returned errors remain user-facing and include the relevant env var, path, and remediation hint. That same error model now applies whether the failure surfaced while building a reqwest client or websocket TLS configuration. ## Tests Pure unit tests in `codex-client` cover env precedence and PEM normalization behavior. Real client construction remains in subprocess tests so the suite can control process env and avoid the macOS seatbelt panic path that motivated the hermetic test split. The subprocess coverage verifies: - `CODEX_CA_CERTIFICATE` precedence over `SSL_CERT_FILE` - fallback to `SSL_CERT_FILE` - single-cert and multi-cert bundles - malformed and empty-file errors - OpenSSL `TRUSTED CERTIFICATE` handling - CRL tolerance for well-formed CRL sections The websocket side is covered by the existing `codex-api` / `codex-core` websocket test suites plus the manual mitmproxy validation above. --------- Co-authored-by: Ivan Zakharchanka <3axap4eHko@gmail.com> Co-authored-by: Codex <noreply@openai.com>
2026-03-12 17:59:26 -07:00
//! These tests intentionally run through `custom_ca_probe` and
//! `build_reqwest_client_for_subprocess_tests` instead of calling the helper in-process. The
//! detailed explanation of what "hermetic" means here lives in `codex_client::custom_ca`; these
//! tests add the process-level half of that contract by scrubbing inherited CA environment
//! variables before each subprocess launch. They still stop at client construction: the
//! assertions here cover CA file selection, PEM parsing, and user-facing errors, not a full TLS
//! handshake.
login: add custom CA support for login flows (#14178) ## Stacked PRs This work is split across three stacked PRs: - #14178: add custom CA support for browser and device-code login flows, docs, and hermetic subprocess tests - #14239: broaden the shared custom CA path from login to other outbound `reqwest` clients across Codex - #14240: extend that shared custom CA handling to secure websocket TLS so websocket connections honor the same CA env vars Review order: #14178, then #14239, then #14240. Supersedes #6864. Thanks to @3axap4eHko for the original implementation and investigation here. Although this version rearranges the code and history significantly, the majority of the credit for this work belongs to them. ## Problem Login flows need to work in enterprise environments where outbound TLS is intercepted by an internal proxy or gateway. In those setups, system root certificates alone are often insufficient to validate the OAuth and device-code endpoints used during login. The change adds a login-specific custom CA loading path, but the important contracts around env precedence, PEM compatibility, test boundaries, and probe-only workarounds need to be explicit so reviewers can understand what behavior is intentional. For users and operators, the behavior is simple: if login needs to trust a custom root CA, set `CODEX_CA_CERTIFICATE` to a PEM file containing one or more certificates. If that variable is unset, login falls back to `SSL_CERT_FILE`. If neither is set, login uses system roots. Invalid or empty PEM files now fail with an error that points back to those environment variables and explains how to recover. ## What This Delivers Users can now make Codex login work behind enterprise TLS interception by pointing `CODEX_CA_CERTIFICATE` at a PEM bundle containing the relevant root certificates. If that variable is unset, login falls back to `SSL_CERT_FILE`, then to system roots. This PR applies that behavior to both browser-based and device-code login flows. It also makes login tolerant of the PEM shapes operators actually have in hand: multi-certificate bundles, OpenSSL `TRUSTED CERTIFICATE` labels, and bundles that include well-formed CRLs. ## Mental model `codex-login` is the place where the login flows construct ad hoc outbound HTTP clients. That makes it the right boundary for a narrow CA policy: look for `CODEX_CA_CERTIFICATE`, fall back to `SSL_CERT_FILE`, load every parseable certificate block in that bundle into a `reqwest::Client`, and fail early with a clear user-facing error if the bundle is unreadable or malformed. The implementation is intentionally pragmatic about PEM input shape. It accepts ordinary certificate bundles, multi-certificate bundles, OpenSSL `TRUSTED CERTIFICATE` labels, and bundles that also contain CRLs. It does not validate a certificate chain or prove a handshake; it only constructs the root store used by login. ## Non-goals This change does not introduce a general-purpose transport abstraction for the rest of the product. It does not validate whether the provided bundle forms a real chain, and it does not add handshake-level integration tests against a live TLS server. It also does not change login state management or OAuth semantics beyond ensuring the existing flows share the same CA-loading rules. ## Tradeoffs The main tradeoff is keeping this logic scoped to login-specific client construction rather than lifting it into a broader shared HTTP layer. That keeps the review surface smaller, but it also means future login-adjacent code must continue to use `build_login_http_client()` or it can silently bypass enterprise CA overrides. The `TRUSTED CERTIFICATE` handling is also intentionally a local compatibility shim. The rustls ecosystem does not currently accept that PEM label upstream, so the code normalizes it locally and trims the OpenSSL `X509_AUX` trailer bytes down to the certificate DER that `reqwest` can consume. ## Architecture `custom_ca.rs` is now the single place that owns login CA behavior. It selects the CA file from the environment, reads it, normalizes PEM label shape where needed, iterates mixed PEM sections with `rustls-pki-types`, ignores CRLs, trims OpenSSL trust metadata when necessary, and returns either a configured `reqwest::Client` or a typed error. The browser login server and the device-code flow both call `build_login_http_client()`, so they share the same trust-store policy. Environment-sensitive tests run through the `login_ca_probe` helper binary because those tests must control process-wide env vars and cannot reliably build a real reqwest client in-process on macOS seatbelt runs. ## Observability The custom CA path logs which environment variable selected the bundle, which file path was loaded, how many certificates were accepted, when `TRUSTED CERTIFICATE` labels were normalized, when CRLs were ignored, and where client construction failed. Returned errors remain user-facing and include the relevant path, env var, and remediation hint. This gives enough signal for three audiences: - users can see why login failed and which env/file caused it - sysadmins can confirm which override actually won - developers can tell whether the failure happened during file read, PEM parsing, certificate registration, or final reqwest client construction ## Tests Pure unit tests stay limited to env precedence and empty-value handling. Real client construction lives in subprocess tests so the suite remains hermetic with respect to process env and macOS sandbox behavior. The subprocess tests verify: - `CODEX_CA_CERTIFICATE` precedence over `SSL_CERT_FILE` - fallback to `SSL_CERT_FILE` - single-certificate and multi-certificate bundles - malformed and empty-bundle errors - OpenSSL `TRUSTED CERTIFICATE` handling - CRL tolerance for well-formed CRL sections The named PEM fixtures under `login/tests/fixtures/` are shared by the tests so their purpose stays reviewable. --------- Co-authored-by: Ivan Zakharchanka <3axap4eHko@gmail.com> Co-authored-by: Codex <noreply@openai.com>
2026-03-12 17:14:54 -07:00
use codex_utils_cargo_bin::cargo_bin;
use std::fs;
use std::path::Path;
use std::process::Command;
use tempfile::TempDir;
const CODEX_CA_CERT_ENV: &str = "CODEX_CA_CERTIFICATE";
const SSL_CERT_FILE_ENV: &str = "SSL_CERT_FILE";
const TEST_CERT_1: &str = include_str!("fixtures/test-ca.pem");
const TEST_CERT_2: &str = include_str!("fixtures/test-intermediate.pem");
const TRUSTED_TEST_CERT: &str = include_str!("fixtures/test-ca-trusted.pem");
fn write_cert_file(temp_dir: &TempDir, name: &str, contents: &str) -> std::path::PathBuf {
let path = temp_dir.path().join(name);
fs::write(&path, contents).unwrap_or_else(|error| {
panic!("write cert fixture failed for {}: {error}", path.display())
});
path
}
fn run_probe(envs: &[(&str, &Path)]) -> std::process::Output {
let mut cmd = Command::new(
client: extend custom CA handling across HTTPS and websocket clients (#14239) ## Stacked PRs This work is now effectively split across two steps: - #14178: add custom CA support for browser and device-code login flows, docs, and hermetic subprocess tests - #14239: extend that shared custom CA handling across Codex HTTPS clients and secure websocket TLS Note: #14240 was merged into this branch while it was stacked on top of this PR. This PR now subsumes that websocket follow-up and should be treated as the combined change. Builds on top of #14178. ## Problem Custom CA support landed first in the login path, but the real requirement is broader. Codex constructs outbound TLS clients in multiple places, and both HTTPS and secure websocket paths can fail behind enterprise TLS interception if they do not honor `CODEX_CA_CERTIFICATE` or `SSL_CERT_FILE` consistently. This PR broadens the shared custom-CA logic beyond login and applies the same policy to websocket TLS, so the enterprise-proxy story is no longer split between “HTTPS works” and “websockets still fail”. ## What This Delivers Custom CA support is no longer limited to login. Codex outbound HTTPS clients and secure websocket connections can now honor the same `CODEX_CA_CERTIFICATE` / `SSL_CERT_FILE` configuration, so enterprise proxy/intercept setups work more consistently end-to-end. For users and operators, nothing new needs to be configured beyond the same CA env vars introduced in #14178. The change is that more of Codex now respects them, including websocket-backed flows that were previously still using default trust roots. I also manually validated the proxy path locally with mitmproxy using: `CODEX_CA_CERTIFICATE=~/.mitmproxy/mitmproxy-ca-cert.pem HTTPS_PROXY=http://127.0.0.1:8080 just codex` with mitmproxy installed via `brew install mitmproxy` and configured as the macOS system proxy. ## Mental model `codex-client` is now the owner of shared custom-CA policy for outbound TLS client construction. Reqwest callers start from the builder configuration they already need, then pass that builder through `build_reqwest_client_with_custom_ca(...)`. Websocket callers ask the same module for a rustls client config when a custom CA bundle is configured. The env precedence is the same everywhere: - `CODEX_CA_CERTIFICATE` wins - otherwise fall back to `SSL_CERT_FILE` - otherwise use system roots The helper is intentionally narrow. It loads every usable certificate from the configured PEM bundle into the appropriate root store and returns either a configured transport or a typed error that explains what went wrong. ## Non-goals This does not add handshake-level integration tests against a live TLS endpoint. It does not validate that the configured bundle forms a meaningful certificate chain. It also does not try to force every transport in the repo through one abstraction; it extends the shared CA policy across the reqwest and websocket paths that actually needed it. ## Tradeoffs The main tradeoff is centralizing CA behavior in `codex-client` while still leaving adoption up to call sites. That keeps the implementation additive and reviewable, but it means the rule "outbound Codex TLS that should honor enterprise roots must use the shared helper" is still partly enforced socially rather than by types. For websockets, the shared helper only builds an explicit rustls config when a custom CA bundle is configured. When no override env var is set, websocket callers still use their ordinary default connector path. ## Architecture `codex-client::custom_ca` now owns CA bundle selection, PEM normalization, mixed-section parsing, certificate extraction, typed CA-loading errors, and optional rustls client-config construction for websocket TLS. The affected consumers now call into that shared helper directly rather than carrying login-local CA behavior: - backend-client - cloud-tasks - RMCP client paths that use `reqwest` - TUI voice HTTP paths - `codex-core` default reqwest client construction - `codex-api` websocket clients for both responses and realtime websocket connections The subprocess CA probe, env-sensitive integration tests, and shared PEM fixtures also live in `codex-client`, which is now the actual owner of the behavior they exercise. ## Observability The shared CA path logs: - which environment variable selected the bundle - which path was loaded - how many certificates were accepted - when `TRUSTED CERTIFICATE` labels were normalized - when CRLs were ignored - where client construction failed Returned errors remain user-facing and include the relevant env var, path, and remediation hint. That same error model now applies whether the failure surfaced while building a reqwest client or websocket TLS configuration. ## Tests Pure unit tests in `codex-client` cover env precedence and PEM normalization behavior. Real client construction remains in subprocess tests so the suite can control process env and avoid the macOS seatbelt panic path that motivated the hermetic test split. The subprocess coverage verifies: - `CODEX_CA_CERTIFICATE` precedence over `SSL_CERT_FILE` - fallback to `SSL_CERT_FILE` - single-cert and multi-cert bundles - malformed and empty-file errors - OpenSSL `TRUSTED CERTIFICATE` handling - CRL tolerance for well-formed CRL sections The websocket side is covered by the existing `codex-api` / `codex-core` websocket test suites plus the manual mitmproxy validation above. --------- Co-authored-by: Ivan Zakharchanka <3axap4eHko@gmail.com> Co-authored-by: Codex <noreply@openai.com>
2026-03-12 17:59:26 -07:00
cargo_bin("custom_ca_probe")
.unwrap_or_else(|error| panic!("failed to locate custom_ca_probe: {error}")),
login: add custom CA support for login flows (#14178) ## Stacked PRs This work is split across three stacked PRs: - #14178: add custom CA support for browser and device-code login flows, docs, and hermetic subprocess tests - #14239: broaden the shared custom CA path from login to other outbound `reqwest` clients across Codex - #14240: extend that shared custom CA handling to secure websocket TLS so websocket connections honor the same CA env vars Review order: #14178, then #14239, then #14240. Supersedes #6864. Thanks to @3axap4eHko for the original implementation and investigation here. Although this version rearranges the code and history significantly, the majority of the credit for this work belongs to them. ## Problem Login flows need to work in enterprise environments where outbound TLS is intercepted by an internal proxy or gateway. In those setups, system root certificates alone are often insufficient to validate the OAuth and device-code endpoints used during login. The change adds a login-specific custom CA loading path, but the important contracts around env precedence, PEM compatibility, test boundaries, and probe-only workarounds need to be explicit so reviewers can understand what behavior is intentional. For users and operators, the behavior is simple: if login needs to trust a custom root CA, set `CODEX_CA_CERTIFICATE` to a PEM file containing one or more certificates. If that variable is unset, login falls back to `SSL_CERT_FILE`. If neither is set, login uses system roots. Invalid or empty PEM files now fail with an error that points back to those environment variables and explains how to recover. ## What This Delivers Users can now make Codex login work behind enterprise TLS interception by pointing `CODEX_CA_CERTIFICATE` at a PEM bundle containing the relevant root certificates. If that variable is unset, login falls back to `SSL_CERT_FILE`, then to system roots. This PR applies that behavior to both browser-based and device-code login flows. It also makes login tolerant of the PEM shapes operators actually have in hand: multi-certificate bundles, OpenSSL `TRUSTED CERTIFICATE` labels, and bundles that include well-formed CRLs. ## Mental model `codex-login` is the place where the login flows construct ad hoc outbound HTTP clients. That makes it the right boundary for a narrow CA policy: look for `CODEX_CA_CERTIFICATE`, fall back to `SSL_CERT_FILE`, load every parseable certificate block in that bundle into a `reqwest::Client`, and fail early with a clear user-facing error if the bundle is unreadable or malformed. The implementation is intentionally pragmatic about PEM input shape. It accepts ordinary certificate bundles, multi-certificate bundles, OpenSSL `TRUSTED CERTIFICATE` labels, and bundles that also contain CRLs. It does not validate a certificate chain or prove a handshake; it only constructs the root store used by login. ## Non-goals This change does not introduce a general-purpose transport abstraction for the rest of the product. It does not validate whether the provided bundle forms a real chain, and it does not add handshake-level integration tests against a live TLS server. It also does not change login state management or OAuth semantics beyond ensuring the existing flows share the same CA-loading rules. ## Tradeoffs The main tradeoff is keeping this logic scoped to login-specific client construction rather than lifting it into a broader shared HTTP layer. That keeps the review surface smaller, but it also means future login-adjacent code must continue to use `build_login_http_client()` or it can silently bypass enterprise CA overrides. The `TRUSTED CERTIFICATE` handling is also intentionally a local compatibility shim. The rustls ecosystem does not currently accept that PEM label upstream, so the code normalizes it locally and trims the OpenSSL `X509_AUX` trailer bytes down to the certificate DER that `reqwest` can consume. ## Architecture `custom_ca.rs` is now the single place that owns login CA behavior. It selects the CA file from the environment, reads it, normalizes PEM label shape where needed, iterates mixed PEM sections with `rustls-pki-types`, ignores CRLs, trims OpenSSL trust metadata when necessary, and returns either a configured `reqwest::Client` or a typed error. The browser login server and the device-code flow both call `build_login_http_client()`, so they share the same trust-store policy. Environment-sensitive tests run through the `login_ca_probe` helper binary because those tests must control process-wide env vars and cannot reliably build a real reqwest client in-process on macOS seatbelt runs. ## Observability The custom CA path logs which environment variable selected the bundle, which file path was loaded, how many certificates were accepted, when `TRUSTED CERTIFICATE` labels were normalized, when CRLs were ignored, and where client construction failed. Returned errors remain user-facing and include the relevant path, env var, and remediation hint. This gives enough signal for three audiences: - users can see why login failed and which env/file caused it - sysadmins can confirm which override actually won - developers can tell whether the failure happened during file read, PEM parsing, certificate registration, or final reqwest client construction ## Tests Pure unit tests stay limited to env precedence and empty-value handling. Real client construction lives in subprocess tests so the suite remains hermetic with respect to process env and macOS sandbox behavior. The subprocess tests verify: - `CODEX_CA_CERTIFICATE` precedence over `SSL_CERT_FILE` - fallback to `SSL_CERT_FILE` - single-certificate and multi-certificate bundles - malformed and empty-bundle errors - OpenSSL `TRUSTED CERTIFICATE` handling - CRL tolerance for well-formed CRL sections The named PEM fixtures under `login/tests/fixtures/` are shared by the tests so their purpose stays reviewable. --------- Co-authored-by: Ivan Zakharchanka <3axap4eHko@gmail.com> Co-authored-by: Codex <noreply@openai.com>
2026-03-12 17:14:54 -07:00
);
// `Command` inherits the parent environment by default, so scrub CA-related variables first or
// these tests can accidentally pass/fail based on the developer shell or CI runner.
cmd.env_remove(CODEX_CA_CERT_ENV);
cmd.env_remove(SSL_CERT_FILE_ENV);
for (key, value) in envs {
cmd.env(key, value);
}
cmd.output()
client: extend custom CA handling across HTTPS and websocket clients (#14239) ## Stacked PRs This work is now effectively split across two steps: - #14178: add custom CA support for browser and device-code login flows, docs, and hermetic subprocess tests - #14239: extend that shared custom CA handling across Codex HTTPS clients and secure websocket TLS Note: #14240 was merged into this branch while it was stacked on top of this PR. This PR now subsumes that websocket follow-up and should be treated as the combined change. Builds on top of #14178. ## Problem Custom CA support landed first in the login path, but the real requirement is broader. Codex constructs outbound TLS clients in multiple places, and both HTTPS and secure websocket paths can fail behind enterprise TLS interception if they do not honor `CODEX_CA_CERTIFICATE` or `SSL_CERT_FILE` consistently. This PR broadens the shared custom-CA logic beyond login and applies the same policy to websocket TLS, so the enterprise-proxy story is no longer split between “HTTPS works” and “websockets still fail”. ## What This Delivers Custom CA support is no longer limited to login. Codex outbound HTTPS clients and secure websocket connections can now honor the same `CODEX_CA_CERTIFICATE` / `SSL_CERT_FILE` configuration, so enterprise proxy/intercept setups work more consistently end-to-end. For users and operators, nothing new needs to be configured beyond the same CA env vars introduced in #14178. The change is that more of Codex now respects them, including websocket-backed flows that were previously still using default trust roots. I also manually validated the proxy path locally with mitmproxy using: `CODEX_CA_CERTIFICATE=~/.mitmproxy/mitmproxy-ca-cert.pem HTTPS_PROXY=http://127.0.0.1:8080 just codex` with mitmproxy installed via `brew install mitmproxy` and configured as the macOS system proxy. ## Mental model `codex-client` is now the owner of shared custom-CA policy for outbound TLS client construction. Reqwest callers start from the builder configuration they already need, then pass that builder through `build_reqwest_client_with_custom_ca(...)`. Websocket callers ask the same module for a rustls client config when a custom CA bundle is configured. The env precedence is the same everywhere: - `CODEX_CA_CERTIFICATE` wins - otherwise fall back to `SSL_CERT_FILE` - otherwise use system roots The helper is intentionally narrow. It loads every usable certificate from the configured PEM bundle into the appropriate root store and returns either a configured transport or a typed error that explains what went wrong. ## Non-goals This does not add handshake-level integration tests against a live TLS endpoint. It does not validate that the configured bundle forms a meaningful certificate chain. It also does not try to force every transport in the repo through one abstraction; it extends the shared CA policy across the reqwest and websocket paths that actually needed it. ## Tradeoffs The main tradeoff is centralizing CA behavior in `codex-client` while still leaving adoption up to call sites. That keeps the implementation additive and reviewable, but it means the rule "outbound Codex TLS that should honor enterprise roots must use the shared helper" is still partly enforced socially rather than by types. For websockets, the shared helper only builds an explicit rustls config when a custom CA bundle is configured. When no override env var is set, websocket callers still use their ordinary default connector path. ## Architecture `codex-client::custom_ca` now owns CA bundle selection, PEM normalization, mixed-section parsing, certificate extraction, typed CA-loading errors, and optional rustls client-config construction for websocket TLS. The affected consumers now call into that shared helper directly rather than carrying login-local CA behavior: - backend-client - cloud-tasks - RMCP client paths that use `reqwest` - TUI voice HTTP paths - `codex-core` default reqwest client construction - `codex-api` websocket clients for both responses and realtime websocket connections The subprocess CA probe, env-sensitive integration tests, and shared PEM fixtures also live in `codex-client`, which is now the actual owner of the behavior they exercise. ## Observability The shared CA path logs: - which environment variable selected the bundle - which path was loaded - how many certificates were accepted - when `TRUSTED CERTIFICATE` labels were normalized - when CRLs were ignored - where client construction failed Returned errors remain user-facing and include the relevant env var, path, and remediation hint. That same error model now applies whether the failure surfaced while building a reqwest client or websocket TLS configuration. ## Tests Pure unit tests in `codex-client` cover env precedence and PEM normalization behavior. Real client construction remains in subprocess tests so the suite can control process env and avoid the macOS seatbelt panic path that motivated the hermetic test split. The subprocess coverage verifies: - `CODEX_CA_CERTIFICATE` precedence over `SSL_CERT_FILE` - fallback to `SSL_CERT_FILE` - single-cert and multi-cert bundles - malformed and empty-file errors - OpenSSL `TRUSTED CERTIFICATE` handling - CRL tolerance for well-formed CRL sections The websocket side is covered by the existing `codex-api` / `codex-core` websocket test suites plus the manual mitmproxy validation above. --------- Co-authored-by: Ivan Zakharchanka <3axap4eHko@gmail.com> Co-authored-by: Codex <noreply@openai.com>
2026-03-12 17:59:26 -07:00
.unwrap_or_else(|error| panic!("failed to run custom_ca_probe: {error}"))
login: add custom CA support for login flows (#14178) ## Stacked PRs This work is split across three stacked PRs: - #14178: add custom CA support for browser and device-code login flows, docs, and hermetic subprocess tests - #14239: broaden the shared custom CA path from login to other outbound `reqwest` clients across Codex - #14240: extend that shared custom CA handling to secure websocket TLS so websocket connections honor the same CA env vars Review order: #14178, then #14239, then #14240. Supersedes #6864. Thanks to @3axap4eHko for the original implementation and investigation here. Although this version rearranges the code and history significantly, the majority of the credit for this work belongs to them. ## Problem Login flows need to work in enterprise environments where outbound TLS is intercepted by an internal proxy or gateway. In those setups, system root certificates alone are often insufficient to validate the OAuth and device-code endpoints used during login. The change adds a login-specific custom CA loading path, but the important contracts around env precedence, PEM compatibility, test boundaries, and probe-only workarounds need to be explicit so reviewers can understand what behavior is intentional. For users and operators, the behavior is simple: if login needs to trust a custom root CA, set `CODEX_CA_CERTIFICATE` to a PEM file containing one or more certificates. If that variable is unset, login falls back to `SSL_CERT_FILE`. If neither is set, login uses system roots. Invalid or empty PEM files now fail with an error that points back to those environment variables and explains how to recover. ## What This Delivers Users can now make Codex login work behind enterprise TLS interception by pointing `CODEX_CA_CERTIFICATE` at a PEM bundle containing the relevant root certificates. If that variable is unset, login falls back to `SSL_CERT_FILE`, then to system roots. This PR applies that behavior to both browser-based and device-code login flows. It also makes login tolerant of the PEM shapes operators actually have in hand: multi-certificate bundles, OpenSSL `TRUSTED CERTIFICATE` labels, and bundles that include well-formed CRLs. ## Mental model `codex-login` is the place where the login flows construct ad hoc outbound HTTP clients. That makes it the right boundary for a narrow CA policy: look for `CODEX_CA_CERTIFICATE`, fall back to `SSL_CERT_FILE`, load every parseable certificate block in that bundle into a `reqwest::Client`, and fail early with a clear user-facing error if the bundle is unreadable or malformed. The implementation is intentionally pragmatic about PEM input shape. It accepts ordinary certificate bundles, multi-certificate bundles, OpenSSL `TRUSTED CERTIFICATE` labels, and bundles that also contain CRLs. It does not validate a certificate chain or prove a handshake; it only constructs the root store used by login. ## Non-goals This change does not introduce a general-purpose transport abstraction for the rest of the product. It does not validate whether the provided bundle forms a real chain, and it does not add handshake-level integration tests against a live TLS server. It also does not change login state management or OAuth semantics beyond ensuring the existing flows share the same CA-loading rules. ## Tradeoffs The main tradeoff is keeping this logic scoped to login-specific client construction rather than lifting it into a broader shared HTTP layer. That keeps the review surface smaller, but it also means future login-adjacent code must continue to use `build_login_http_client()` or it can silently bypass enterprise CA overrides. The `TRUSTED CERTIFICATE` handling is also intentionally a local compatibility shim. The rustls ecosystem does not currently accept that PEM label upstream, so the code normalizes it locally and trims the OpenSSL `X509_AUX` trailer bytes down to the certificate DER that `reqwest` can consume. ## Architecture `custom_ca.rs` is now the single place that owns login CA behavior. It selects the CA file from the environment, reads it, normalizes PEM label shape where needed, iterates mixed PEM sections with `rustls-pki-types`, ignores CRLs, trims OpenSSL trust metadata when necessary, and returns either a configured `reqwest::Client` or a typed error. The browser login server and the device-code flow both call `build_login_http_client()`, so they share the same trust-store policy. Environment-sensitive tests run through the `login_ca_probe` helper binary because those tests must control process-wide env vars and cannot reliably build a real reqwest client in-process on macOS seatbelt runs. ## Observability The custom CA path logs which environment variable selected the bundle, which file path was loaded, how many certificates were accepted, when `TRUSTED CERTIFICATE` labels were normalized, when CRLs were ignored, and where client construction failed. Returned errors remain user-facing and include the relevant path, env var, and remediation hint. This gives enough signal for three audiences: - users can see why login failed and which env/file caused it - sysadmins can confirm which override actually won - developers can tell whether the failure happened during file read, PEM parsing, certificate registration, or final reqwest client construction ## Tests Pure unit tests stay limited to env precedence and empty-value handling. Real client construction lives in subprocess tests so the suite remains hermetic with respect to process env and macOS sandbox behavior. The subprocess tests verify: - `CODEX_CA_CERTIFICATE` precedence over `SSL_CERT_FILE` - fallback to `SSL_CERT_FILE` - single-certificate and multi-certificate bundles - malformed and empty-bundle errors - OpenSSL `TRUSTED CERTIFICATE` handling - CRL tolerance for well-formed CRL sections The named PEM fixtures under `login/tests/fixtures/` are shared by the tests so their purpose stays reviewable. --------- Co-authored-by: Ivan Zakharchanka <3axap4eHko@gmail.com> Co-authored-by: Codex <noreply@openai.com>
2026-03-12 17:14:54 -07:00
}
#[test]
fn uses_codex_ca_cert_env() {
let temp_dir = TempDir::new().expect("tempdir");
let cert_path = write_cert_file(&temp_dir, "ca.pem", TEST_CERT_1);
let output = run_probe(&[(CODEX_CA_CERT_ENV, cert_path.as_path())]);
assert!(output.status.success());
}
#[test]
fn falls_back_to_ssl_cert_file() {
let temp_dir = TempDir::new().expect("tempdir");
let cert_path = write_cert_file(&temp_dir, "ssl.pem", TEST_CERT_1);
let output = run_probe(&[(SSL_CERT_FILE_ENV, cert_path.as_path())]);
assert!(output.status.success());
}
#[test]
fn prefers_codex_ca_cert_over_ssl_cert_file() {
let temp_dir = TempDir::new().expect("tempdir");
let cert_path = write_cert_file(&temp_dir, "ca.pem", TEST_CERT_1);
let bad_path = write_cert_file(&temp_dir, "bad.pem", "");
let output = run_probe(&[
(CODEX_CA_CERT_ENV, cert_path.as_path()),
(SSL_CERT_FILE_ENV, bad_path.as_path()),
]);
assert!(output.status.success());
}
#[test]
fn handles_multi_certificate_bundle() {
let temp_dir = TempDir::new().expect("tempdir");
let bundle = format!("{TEST_CERT_1}\n{TEST_CERT_2}");
let cert_path = write_cert_file(&temp_dir, "bundle.pem", &bundle);
let output = run_probe(&[(CODEX_CA_CERT_ENV, cert_path.as_path())]);
assert!(output.status.success());
}
#[test]
fn rejects_empty_pem_file_with_hint() {
let temp_dir = TempDir::new().expect("tempdir");
let cert_path = write_cert_file(&temp_dir, "empty.pem", "");
let output = run_probe(&[(CODEX_CA_CERT_ENV, cert_path.as_path())]);
assert!(!output.status.success());
let stderr = String::from_utf8_lossy(&output.stderr);
assert!(stderr.contains("no certificates found in PEM file"));
assert!(stderr.contains("CODEX_CA_CERTIFICATE"));
assert!(stderr.contains("SSL_CERT_FILE"));
}
#[test]
fn rejects_malformed_pem_with_hint() {
let temp_dir = TempDir::new().expect("tempdir");
let cert_path = write_cert_file(
&temp_dir,
"malformed.pem",
"-----BEGIN CERTIFICATE-----\nMIIBroken",
);
let output = run_probe(&[(CODEX_CA_CERT_ENV, cert_path.as_path())]);
assert!(!output.status.success());
let stderr = String::from_utf8_lossy(&output.stderr);
assert!(stderr.contains("failed to parse PEM file"));
assert!(stderr.contains("CODEX_CA_CERTIFICATE"));
assert!(stderr.contains("SSL_CERT_FILE"));
}
#[test]
fn accepts_openssl_trusted_certificate() {
let temp_dir = TempDir::new().expect("tempdir");
let cert_path = write_cert_file(&temp_dir, "trusted.pem", TRUSTED_TEST_CERT);
let output = run_probe(&[(CODEX_CA_CERT_ENV, cert_path.as_path())]);
assert!(output.status.success());
}
#[test]
fn accepts_bundle_with_crl() {
let temp_dir = TempDir::new().expect("tempdir");
let crl = "-----BEGIN X509 CRL-----\nMIIC\n-----END X509 CRL-----";
let bundle = format!("{TEST_CERT_1}\n{crl}");
let cert_path = write_cert_file(&temp_dir, "bundle_crl.pem", &bundle);
let output = run_probe(&[(CODEX_CA_CERT_ENV, cert_path.as_path())]);
assert!(output.status.success());
}