login: add custom CA support for login flows (#14178)
## Stacked PRs
This work is split across three stacked PRs:
- #14178: add custom CA support for browser and device-code login flows,
docs, and hermetic subprocess tests
- #14239: broaden the shared custom CA path from login to other outbound
`reqwest` clients across Codex
- #14240: extend that shared custom CA handling to secure websocket TLS
so websocket connections honor the same CA env vars
Review order: #14178, then #14239, then #14240.
Supersedes #6864.
Thanks to @3axap4eHko for the original implementation and investigation
here. Although this version rearranges the code and history
significantly, the majority of the credit for this work belongs to them.
## Problem
Login flows need to work in enterprise environments where outbound TLS
is intercepted by an internal proxy or gateway. In those setups, system
root certificates alone are often insufficient to validate the OAuth and
device-code endpoints used during login. The change adds a
login-specific custom CA loading path, but the important contracts
around env precedence, PEM compatibility, test boundaries, and
probe-only workarounds need to be explicit so reviewers can understand
what behavior is intentional.
For users and operators, the behavior is simple: if login needs to trust
a custom root CA, set `CODEX_CA_CERTIFICATE` to a PEM file containing
one or more certificates. If that variable is unset, login falls back to
`SSL_CERT_FILE`. If neither is set, login uses system roots. Invalid or
empty PEM files now fail with an error that points back to those
environment variables and explains how to recover.
## What This Delivers
Users can now make Codex login work behind enterprise TLS interception
by pointing `CODEX_CA_CERTIFICATE` at a PEM bundle containing the
relevant root certificates. If that variable is unset, login falls back
to `SSL_CERT_FILE`, then to system roots.
This PR applies that behavior to both browser-based and device-code
login flows. It also makes login tolerant of the PEM shapes operators
actually have in hand: multi-certificate bundles, OpenSSL `TRUSTED
CERTIFICATE` labels, and bundles that include well-formed CRLs.
## Mental model
`codex-login` is the place where the login flows construct ad hoc
outbound HTTP clients. That makes it the right boundary for a narrow CA
policy: look for `CODEX_CA_CERTIFICATE`, fall back to `SSL_CERT_FILE`,
load every parseable certificate block in that bundle into a
`reqwest::Client`, and fail early with a clear user-facing error if the
bundle is unreadable or malformed.
The implementation is intentionally pragmatic about PEM input shape. It
accepts ordinary certificate bundles, multi-certificate bundles, OpenSSL
`TRUSTED CERTIFICATE` labels, and bundles that also contain CRLs. It
does not validate a certificate chain or prove a handshake; it only
constructs the root store used by login.
## Non-goals
This change does not introduce a general-purpose transport abstraction
for the rest of the product. It does not validate whether the provided
bundle forms a real chain, and it does not add handshake-level
integration tests against a live TLS server. It also does not change
login state management or OAuth semantics beyond ensuring the existing
flows share the same CA-loading rules.
## Tradeoffs
The main tradeoff is keeping this logic scoped to login-specific client
construction rather than lifting it into a broader shared HTTP layer.
That keeps the review surface smaller, but it also means future
login-adjacent code must continue to use `build_login_http_client()` or
it can silently bypass enterprise CA overrides.
The `TRUSTED CERTIFICATE` handling is also intentionally a local
compatibility shim. The rustls ecosystem does not currently accept that
PEM label upstream, so the code normalizes it locally and trims the
OpenSSL `X509_AUX` trailer bytes down to the certificate DER that
`reqwest` can consume.
## Architecture
`custom_ca.rs` is now the single place that owns login CA behavior. It
selects the CA file from the environment, reads it, normalizes PEM label
shape where needed, iterates mixed PEM sections with `rustls-pki-types`,
ignores CRLs, trims OpenSSL trust metadata when necessary, and returns
either a configured `reqwest::Client` or a typed error.
The browser login server and the device-code flow both call
`build_login_http_client()`, so they share the same trust-store policy.
Environment-sensitive tests run through the `login_ca_probe` helper
binary because those tests must control process-wide env vars and cannot
reliably build a real reqwest client in-process on macOS seatbelt runs.
## Observability
The custom CA path logs which environment variable selected the bundle,
which file path was loaded, how many certificates were accepted, when
`TRUSTED CERTIFICATE` labels were normalized, when CRLs were ignored,
and where client construction failed. Returned errors remain user-facing
and include the relevant path, env var, and remediation hint.
This gives enough signal for three audiences:
- users can see why login failed and which env/file caused it
- sysadmins can confirm which override actually won
- developers can tell whether the failure happened during file read, PEM
parsing, certificate registration, or final reqwest client construction
## Tests
Pure unit tests stay limited to env precedence and empty-value handling.
Real client construction lives in subprocess tests so the suite remains
hermetic with respect to process env and macOS sandbox behavior.
The subprocess tests verify:
- `CODEX_CA_CERTIFICATE` precedence over `SSL_CERT_FILE`
- fallback to `SSL_CERT_FILE`
- single-certificate and multi-certificate bundles
- malformed and empty-bundle errors
- OpenSSL `TRUSTED CERTIFICATE` handling
- CRL tolerance for well-formed CRL sections
The named PEM fixtures under `login/tests/fixtures/` are shared by the
tests so their purpose stays reviewable.
---------
Co-authored-by: Ivan Zakharchanka <3axap4eHko@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-03-12 17:14:54 -07:00
|
|
|
//! Subprocess coverage for custom CA behavior that must build a real reqwest client.
|
|
|
|
|
//!
|
client: extend custom CA handling across HTTPS and websocket clients (#14239)
## Stacked PRs
This work is now effectively split across two steps:
- #14178: add custom CA support for browser and device-code login flows,
docs, and hermetic subprocess tests
- #14239: extend that shared custom CA handling across Codex HTTPS
clients and secure websocket TLS
Note: #14240 was merged into this branch while it was stacked on top of
this PR. This PR now subsumes that websocket follow-up and should be
treated as the combined change.
Builds on top of #14178.
## Problem
Custom CA support landed first in the login path, but the real
requirement is broader. Codex constructs outbound TLS clients in
multiple places, and both HTTPS and secure websocket paths can fail
behind enterprise TLS interception if they do not honor
`CODEX_CA_CERTIFICATE` or `SSL_CERT_FILE` consistently.
This PR broadens the shared custom-CA logic beyond login and applies the
same policy to websocket TLS, so the enterprise-proxy story is no longer
split between “HTTPS works” and “websockets still fail”.
## What This Delivers
Custom CA support is no longer limited to login. Codex outbound HTTPS
clients and secure websocket connections can now honor the same
`CODEX_CA_CERTIFICATE` / `SSL_CERT_FILE` configuration, so enterprise
proxy/intercept setups work more consistently end-to-end.
For users and operators, nothing new needs to be configured beyond the
same CA env vars introduced in #14178. The change is that more of Codex
now respects them, including websocket-backed flows that were previously
still using default trust roots.
I also manually validated the proxy path locally with mitmproxy using:
`CODEX_CA_CERTIFICATE=~/.mitmproxy/mitmproxy-ca-cert.pem
HTTPS_PROXY=http://127.0.0.1:8080 just codex`
with mitmproxy installed via `brew install mitmproxy` and configured as
the macOS system proxy.
## Mental model
`codex-client` is now the owner of shared custom-CA policy for outbound
TLS client construction. Reqwest callers start from the builder
configuration they already need, then pass that builder through
`build_reqwest_client_with_custom_ca(...)`. Websocket callers ask the
same module for a rustls client config when a custom CA bundle is
configured.
The env precedence is the same everywhere:
- `CODEX_CA_CERTIFICATE` wins
- otherwise fall back to `SSL_CERT_FILE`
- otherwise use system roots
The helper is intentionally narrow. It loads every usable certificate
from the configured PEM bundle into the appropriate root store and
returns either a configured transport or a typed error that explains
what went wrong.
## Non-goals
This does not add handshake-level integration tests against a live TLS
endpoint. It does not validate that the configured bundle forms a
meaningful certificate chain. It also does not try to force every
transport in the repo through one abstraction; it extends the shared CA
policy across the reqwest and websocket paths that actually needed it.
## Tradeoffs
The main tradeoff is centralizing CA behavior in `codex-client` while
still leaving adoption up to call sites. That keeps the implementation
additive and reviewable, but it means the rule "outbound Codex TLS that
should honor enterprise roots must use the shared helper" is still
partly enforced socially rather than by types.
For websockets, the shared helper only builds an explicit rustls config
when a custom CA bundle is configured. When no override env var is set,
websocket callers still use their ordinary default connector path.
## Architecture
`codex-client::custom_ca` now owns CA bundle selection, PEM
normalization, mixed-section parsing, certificate extraction, typed
CA-loading errors, and optional rustls client-config construction for
websocket TLS.
The affected consumers now call into that shared helper directly rather
than carrying login-local CA behavior:
- backend-client
- cloud-tasks
- RMCP client paths that use `reqwest`
- TUI voice HTTP paths
- `codex-core` default reqwest client construction
- `codex-api` websocket clients for both responses and realtime
websocket connections
The subprocess CA probe, env-sensitive integration tests, and shared PEM
fixtures also live in `codex-client`, which is now the actual owner of
the behavior they exercise.
## Observability
The shared CA path logs:
- which environment variable selected the bundle
- which path was loaded
- how many certificates were accepted
- when `TRUSTED CERTIFICATE` labels were normalized
- when CRLs were ignored
- where client construction failed
Returned errors remain user-facing and include the relevant env var,
path, and remediation hint. That same error model now applies whether
the failure surfaced while building a reqwest client or websocket TLS
configuration.
## Tests
Pure unit tests in `codex-client` cover env precedence and PEM
normalization behavior. Real client construction remains in subprocess
tests so the suite can control process env and avoid the macOS seatbelt
panic path that motivated the hermetic test split.
The subprocess coverage verifies:
- `CODEX_CA_CERTIFICATE` precedence over `SSL_CERT_FILE`
- fallback to `SSL_CERT_FILE`
- single-cert and multi-cert bundles
- malformed and empty-file errors
- OpenSSL `TRUSTED CERTIFICATE` handling
- CRL tolerance for well-formed CRL sections
The websocket side is covered by the existing `codex-api` / `codex-core`
websocket test suites plus the manual mitmproxy validation above.
---------
Co-authored-by: Ivan Zakharchanka <3axap4eHko@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-03-12 17:59:26 -07:00
|
|
|
//! These tests intentionally run through `custom_ca_probe` and
|
|
|
|
|
//! `build_reqwest_client_for_subprocess_tests` instead of calling the helper in-process. The
|
|
|
|
|
//! detailed explanation of what "hermetic" means here lives in `codex_client::custom_ca`; these
|
|
|
|
|
//! tests add the process-level half of that contract by scrubbing inherited CA environment
|
|
|
|
|
//! variables before each subprocess launch. They still stop at client construction: the
|
|
|
|
|
//! assertions here cover CA file selection, PEM parsing, and user-facing errors, not a full TLS
|
|
|
|
|
//! handshake.
|
login: add custom CA support for login flows (#14178)
## Stacked PRs
This work is split across three stacked PRs:
- #14178: add custom CA support for browser and device-code login flows,
docs, and hermetic subprocess tests
- #14239: broaden the shared custom CA path from login to other outbound
`reqwest` clients across Codex
- #14240: extend that shared custom CA handling to secure websocket TLS
so websocket connections honor the same CA env vars
Review order: #14178, then #14239, then #14240.
Supersedes #6864.
Thanks to @3axap4eHko for the original implementation and investigation
here. Although this version rearranges the code and history
significantly, the majority of the credit for this work belongs to them.
## Problem
Login flows need to work in enterprise environments where outbound TLS
is intercepted by an internal proxy or gateway. In those setups, system
root certificates alone are often insufficient to validate the OAuth and
device-code endpoints used during login. The change adds a
login-specific custom CA loading path, but the important contracts
around env precedence, PEM compatibility, test boundaries, and
probe-only workarounds need to be explicit so reviewers can understand
what behavior is intentional.
For users and operators, the behavior is simple: if login needs to trust
a custom root CA, set `CODEX_CA_CERTIFICATE` to a PEM file containing
one or more certificates. If that variable is unset, login falls back to
`SSL_CERT_FILE`. If neither is set, login uses system roots. Invalid or
empty PEM files now fail with an error that points back to those
environment variables and explains how to recover.
## What This Delivers
Users can now make Codex login work behind enterprise TLS interception
by pointing `CODEX_CA_CERTIFICATE` at a PEM bundle containing the
relevant root certificates. If that variable is unset, login falls back
to `SSL_CERT_FILE`, then to system roots.
This PR applies that behavior to both browser-based and device-code
login flows. It also makes login tolerant of the PEM shapes operators
actually have in hand: multi-certificate bundles, OpenSSL `TRUSTED
CERTIFICATE` labels, and bundles that include well-formed CRLs.
## Mental model
`codex-login` is the place where the login flows construct ad hoc
outbound HTTP clients. That makes it the right boundary for a narrow CA
policy: look for `CODEX_CA_CERTIFICATE`, fall back to `SSL_CERT_FILE`,
load every parseable certificate block in that bundle into a
`reqwest::Client`, and fail early with a clear user-facing error if the
bundle is unreadable or malformed.
The implementation is intentionally pragmatic about PEM input shape. It
accepts ordinary certificate bundles, multi-certificate bundles, OpenSSL
`TRUSTED CERTIFICATE` labels, and bundles that also contain CRLs. It
does not validate a certificate chain or prove a handshake; it only
constructs the root store used by login.
## Non-goals
This change does not introduce a general-purpose transport abstraction
for the rest of the product. It does not validate whether the provided
bundle forms a real chain, and it does not add handshake-level
integration tests against a live TLS server. It also does not change
login state management or OAuth semantics beyond ensuring the existing
flows share the same CA-loading rules.
## Tradeoffs
The main tradeoff is keeping this logic scoped to login-specific client
construction rather than lifting it into a broader shared HTTP layer.
That keeps the review surface smaller, but it also means future
login-adjacent code must continue to use `build_login_http_client()` or
it can silently bypass enterprise CA overrides.
The `TRUSTED CERTIFICATE` handling is also intentionally a local
compatibility shim. The rustls ecosystem does not currently accept that
PEM label upstream, so the code normalizes it locally and trims the
OpenSSL `X509_AUX` trailer bytes down to the certificate DER that
`reqwest` can consume.
## Architecture
`custom_ca.rs` is now the single place that owns login CA behavior. It
selects the CA file from the environment, reads it, normalizes PEM label
shape where needed, iterates mixed PEM sections with `rustls-pki-types`,
ignores CRLs, trims OpenSSL trust metadata when necessary, and returns
either a configured `reqwest::Client` or a typed error.
The browser login server and the device-code flow both call
`build_login_http_client()`, so they share the same trust-store policy.
Environment-sensitive tests run through the `login_ca_probe` helper
binary because those tests must control process-wide env vars and cannot
reliably build a real reqwest client in-process on macOS seatbelt runs.
## Observability
The custom CA path logs which environment variable selected the bundle,
which file path was loaded, how many certificates were accepted, when
`TRUSTED CERTIFICATE` labels were normalized, when CRLs were ignored,
and where client construction failed. Returned errors remain user-facing
and include the relevant path, env var, and remediation hint.
This gives enough signal for three audiences:
- users can see why login failed and which env/file caused it
- sysadmins can confirm which override actually won
- developers can tell whether the failure happened during file read, PEM
parsing, certificate registration, or final reqwest client construction
## Tests
Pure unit tests stay limited to env precedence and empty-value handling.
Real client construction lives in subprocess tests so the suite remains
hermetic with respect to process env and macOS sandbox behavior.
The subprocess tests verify:
- `CODEX_CA_CERTIFICATE` precedence over `SSL_CERT_FILE`
- fallback to `SSL_CERT_FILE`
- single-certificate and multi-certificate bundles
- malformed and empty-bundle errors
- OpenSSL `TRUSTED CERTIFICATE` handling
- CRL tolerance for well-formed CRL sections
The named PEM fixtures under `login/tests/fixtures/` are shared by the
tests so their purpose stays reviewable.
---------
Co-authored-by: Ivan Zakharchanka <3axap4eHko@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-03-12 17:14:54 -07:00
|
|
|
|
|
|
|
|
use codex_utils_cargo_bin::cargo_bin;
|
|
|
|
|
use std::fs;
|
|
|
|
|
use std::path::Path;
|
|
|
|
|
use std::process::Command;
|
|
|
|
|
use tempfile::TempDir;
|
|
|
|
|
|
|
|
|
|
const CODEX_CA_CERT_ENV: &str = "CODEX_CA_CERTIFICATE";
|
|
|
|
|
const SSL_CERT_FILE_ENV: &str = "SSL_CERT_FILE";
|
|
|
|
|
|
|
|
|
|
const TEST_CERT_1: &str = include_str!("fixtures/test-ca.pem");
|
|
|
|
|
const TEST_CERT_2: &str = include_str!("fixtures/test-intermediate.pem");
|
|
|
|
|
const TRUSTED_TEST_CERT: &str = include_str!("fixtures/test-ca-trusted.pem");
|
|
|
|
|
|
|
|
|
|
fn write_cert_file(temp_dir: &TempDir, name: &str, contents: &str) -> std::path::PathBuf {
|
|
|
|
|
let path = temp_dir.path().join(name);
|
|
|
|
|
fs::write(&path, contents).unwrap_or_else(|error| {
|
|
|
|
|
panic!("write cert fixture failed for {}: {error}", path.display())
|
|
|
|
|
});
|
|
|
|
|
path
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
fn run_probe(envs: &[(&str, &Path)]) -> std::process::Output {
|
|
|
|
|
let mut cmd = Command::new(
|
client: extend custom CA handling across HTTPS and websocket clients (#14239)
## Stacked PRs
This work is now effectively split across two steps:
- #14178: add custom CA support for browser and device-code login flows,
docs, and hermetic subprocess tests
- #14239: extend that shared custom CA handling across Codex HTTPS
clients and secure websocket TLS
Note: #14240 was merged into this branch while it was stacked on top of
this PR. This PR now subsumes that websocket follow-up and should be
treated as the combined change.
Builds on top of #14178.
## Problem
Custom CA support landed first in the login path, but the real
requirement is broader. Codex constructs outbound TLS clients in
multiple places, and both HTTPS and secure websocket paths can fail
behind enterprise TLS interception if they do not honor
`CODEX_CA_CERTIFICATE` or `SSL_CERT_FILE` consistently.
This PR broadens the shared custom-CA logic beyond login and applies the
same policy to websocket TLS, so the enterprise-proxy story is no longer
split between “HTTPS works” and “websockets still fail”.
## What This Delivers
Custom CA support is no longer limited to login. Codex outbound HTTPS
clients and secure websocket connections can now honor the same
`CODEX_CA_CERTIFICATE` / `SSL_CERT_FILE` configuration, so enterprise
proxy/intercept setups work more consistently end-to-end.
For users and operators, nothing new needs to be configured beyond the
same CA env vars introduced in #14178. The change is that more of Codex
now respects them, including websocket-backed flows that were previously
still using default trust roots.
I also manually validated the proxy path locally with mitmproxy using:
`CODEX_CA_CERTIFICATE=~/.mitmproxy/mitmproxy-ca-cert.pem
HTTPS_PROXY=http://127.0.0.1:8080 just codex`
with mitmproxy installed via `brew install mitmproxy` and configured as
the macOS system proxy.
## Mental model
`codex-client` is now the owner of shared custom-CA policy for outbound
TLS client construction. Reqwest callers start from the builder
configuration they already need, then pass that builder through
`build_reqwest_client_with_custom_ca(...)`. Websocket callers ask the
same module for a rustls client config when a custom CA bundle is
configured.
The env precedence is the same everywhere:
- `CODEX_CA_CERTIFICATE` wins
- otherwise fall back to `SSL_CERT_FILE`
- otherwise use system roots
The helper is intentionally narrow. It loads every usable certificate
from the configured PEM bundle into the appropriate root store and
returns either a configured transport or a typed error that explains
what went wrong.
## Non-goals
This does not add handshake-level integration tests against a live TLS
endpoint. It does not validate that the configured bundle forms a
meaningful certificate chain. It also does not try to force every
transport in the repo through one abstraction; it extends the shared CA
policy across the reqwest and websocket paths that actually needed it.
## Tradeoffs
The main tradeoff is centralizing CA behavior in `codex-client` while
still leaving adoption up to call sites. That keeps the implementation
additive and reviewable, but it means the rule "outbound Codex TLS that
should honor enterprise roots must use the shared helper" is still
partly enforced socially rather than by types.
For websockets, the shared helper only builds an explicit rustls config
when a custom CA bundle is configured. When no override env var is set,
websocket callers still use their ordinary default connector path.
## Architecture
`codex-client::custom_ca` now owns CA bundle selection, PEM
normalization, mixed-section parsing, certificate extraction, typed
CA-loading errors, and optional rustls client-config construction for
websocket TLS.
The affected consumers now call into that shared helper directly rather
than carrying login-local CA behavior:
- backend-client
- cloud-tasks
- RMCP client paths that use `reqwest`
- TUI voice HTTP paths
- `codex-core` default reqwest client construction
- `codex-api` websocket clients for both responses and realtime
websocket connections
The subprocess CA probe, env-sensitive integration tests, and shared PEM
fixtures also live in `codex-client`, which is now the actual owner of
the behavior they exercise.
## Observability
The shared CA path logs:
- which environment variable selected the bundle
- which path was loaded
- how many certificates were accepted
- when `TRUSTED CERTIFICATE` labels were normalized
- when CRLs were ignored
- where client construction failed
Returned errors remain user-facing and include the relevant env var,
path, and remediation hint. That same error model now applies whether
the failure surfaced while building a reqwest client or websocket TLS
configuration.
## Tests
Pure unit tests in `codex-client` cover env precedence and PEM
normalization behavior. Real client construction remains in subprocess
tests so the suite can control process env and avoid the macOS seatbelt
panic path that motivated the hermetic test split.
The subprocess coverage verifies:
- `CODEX_CA_CERTIFICATE` precedence over `SSL_CERT_FILE`
- fallback to `SSL_CERT_FILE`
- single-cert and multi-cert bundles
- malformed and empty-file errors
- OpenSSL `TRUSTED CERTIFICATE` handling
- CRL tolerance for well-formed CRL sections
The websocket side is covered by the existing `codex-api` / `codex-core`
websocket test suites plus the manual mitmproxy validation above.
---------
Co-authored-by: Ivan Zakharchanka <3axap4eHko@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-03-12 17:59:26 -07:00
|
|
|
cargo_bin("custom_ca_probe")
|
|
|
|
|
.unwrap_or_else(|error| panic!("failed to locate custom_ca_probe: {error}")),
|
login: add custom CA support for login flows (#14178)
## Stacked PRs
This work is split across three stacked PRs:
- #14178: add custom CA support for browser and device-code login flows,
docs, and hermetic subprocess tests
- #14239: broaden the shared custom CA path from login to other outbound
`reqwest` clients across Codex
- #14240: extend that shared custom CA handling to secure websocket TLS
so websocket connections honor the same CA env vars
Review order: #14178, then #14239, then #14240.
Supersedes #6864.
Thanks to @3axap4eHko for the original implementation and investigation
here. Although this version rearranges the code and history
significantly, the majority of the credit for this work belongs to them.
## Problem
Login flows need to work in enterprise environments where outbound TLS
is intercepted by an internal proxy or gateway. In those setups, system
root certificates alone are often insufficient to validate the OAuth and
device-code endpoints used during login. The change adds a
login-specific custom CA loading path, but the important contracts
around env precedence, PEM compatibility, test boundaries, and
probe-only workarounds need to be explicit so reviewers can understand
what behavior is intentional.
For users and operators, the behavior is simple: if login needs to trust
a custom root CA, set `CODEX_CA_CERTIFICATE` to a PEM file containing
one or more certificates. If that variable is unset, login falls back to
`SSL_CERT_FILE`. If neither is set, login uses system roots. Invalid or
empty PEM files now fail with an error that points back to those
environment variables and explains how to recover.
## What This Delivers
Users can now make Codex login work behind enterprise TLS interception
by pointing `CODEX_CA_CERTIFICATE` at a PEM bundle containing the
relevant root certificates. If that variable is unset, login falls back
to `SSL_CERT_FILE`, then to system roots.
This PR applies that behavior to both browser-based and device-code
login flows. It also makes login tolerant of the PEM shapes operators
actually have in hand: multi-certificate bundles, OpenSSL `TRUSTED
CERTIFICATE` labels, and bundles that include well-formed CRLs.
## Mental model
`codex-login` is the place where the login flows construct ad hoc
outbound HTTP clients. That makes it the right boundary for a narrow CA
policy: look for `CODEX_CA_CERTIFICATE`, fall back to `SSL_CERT_FILE`,
load every parseable certificate block in that bundle into a
`reqwest::Client`, and fail early with a clear user-facing error if the
bundle is unreadable or malformed.
The implementation is intentionally pragmatic about PEM input shape. It
accepts ordinary certificate bundles, multi-certificate bundles, OpenSSL
`TRUSTED CERTIFICATE` labels, and bundles that also contain CRLs. It
does not validate a certificate chain or prove a handshake; it only
constructs the root store used by login.
## Non-goals
This change does not introduce a general-purpose transport abstraction
for the rest of the product. It does not validate whether the provided
bundle forms a real chain, and it does not add handshake-level
integration tests against a live TLS server. It also does not change
login state management or OAuth semantics beyond ensuring the existing
flows share the same CA-loading rules.
## Tradeoffs
The main tradeoff is keeping this logic scoped to login-specific client
construction rather than lifting it into a broader shared HTTP layer.
That keeps the review surface smaller, but it also means future
login-adjacent code must continue to use `build_login_http_client()` or
it can silently bypass enterprise CA overrides.
The `TRUSTED CERTIFICATE` handling is also intentionally a local
compatibility shim. The rustls ecosystem does not currently accept that
PEM label upstream, so the code normalizes it locally and trims the
OpenSSL `X509_AUX` trailer bytes down to the certificate DER that
`reqwest` can consume.
## Architecture
`custom_ca.rs` is now the single place that owns login CA behavior. It
selects the CA file from the environment, reads it, normalizes PEM label
shape where needed, iterates mixed PEM sections with `rustls-pki-types`,
ignores CRLs, trims OpenSSL trust metadata when necessary, and returns
either a configured `reqwest::Client` or a typed error.
The browser login server and the device-code flow both call
`build_login_http_client()`, so they share the same trust-store policy.
Environment-sensitive tests run through the `login_ca_probe` helper
binary because those tests must control process-wide env vars and cannot
reliably build a real reqwest client in-process on macOS seatbelt runs.
## Observability
The custom CA path logs which environment variable selected the bundle,
which file path was loaded, how many certificates were accepted, when
`TRUSTED CERTIFICATE` labels were normalized, when CRLs were ignored,
and where client construction failed. Returned errors remain user-facing
and include the relevant path, env var, and remediation hint.
This gives enough signal for three audiences:
- users can see why login failed and which env/file caused it
- sysadmins can confirm which override actually won
- developers can tell whether the failure happened during file read, PEM
parsing, certificate registration, or final reqwest client construction
## Tests
Pure unit tests stay limited to env precedence and empty-value handling.
Real client construction lives in subprocess tests so the suite remains
hermetic with respect to process env and macOS sandbox behavior.
The subprocess tests verify:
- `CODEX_CA_CERTIFICATE` precedence over `SSL_CERT_FILE`
- fallback to `SSL_CERT_FILE`
- single-certificate and multi-certificate bundles
- malformed and empty-bundle errors
- OpenSSL `TRUSTED CERTIFICATE` handling
- CRL tolerance for well-formed CRL sections
The named PEM fixtures under `login/tests/fixtures/` are shared by the
tests so their purpose stays reviewable.
---------
Co-authored-by: Ivan Zakharchanka <3axap4eHko@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-03-12 17:14:54 -07:00
|
|
|
);
|
|
|
|
|
// `Command` inherits the parent environment by default, so scrub CA-related variables first or
|
|
|
|
|
// these tests can accidentally pass/fail based on the developer shell or CI runner.
|
|
|
|
|
cmd.env_remove(CODEX_CA_CERT_ENV);
|
|
|
|
|
cmd.env_remove(SSL_CERT_FILE_ENV);
|
|
|
|
|
for (key, value) in envs {
|
|
|
|
|
cmd.env(key, value);
|
|
|
|
|
}
|
|
|
|
|
cmd.output()
|
client: extend custom CA handling across HTTPS and websocket clients (#14239)
## Stacked PRs
This work is now effectively split across two steps:
- #14178: add custom CA support for browser and device-code login flows,
docs, and hermetic subprocess tests
- #14239: extend that shared custom CA handling across Codex HTTPS
clients and secure websocket TLS
Note: #14240 was merged into this branch while it was stacked on top of
this PR. This PR now subsumes that websocket follow-up and should be
treated as the combined change.
Builds on top of #14178.
## Problem
Custom CA support landed first in the login path, but the real
requirement is broader. Codex constructs outbound TLS clients in
multiple places, and both HTTPS and secure websocket paths can fail
behind enterprise TLS interception if they do not honor
`CODEX_CA_CERTIFICATE` or `SSL_CERT_FILE` consistently.
This PR broadens the shared custom-CA logic beyond login and applies the
same policy to websocket TLS, so the enterprise-proxy story is no longer
split between “HTTPS works” and “websockets still fail”.
## What This Delivers
Custom CA support is no longer limited to login. Codex outbound HTTPS
clients and secure websocket connections can now honor the same
`CODEX_CA_CERTIFICATE` / `SSL_CERT_FILE` configuration, so enterprise
proxy/intercept setups work more consistently end-to-end.
For users and operators, nothing new needs to be configured beyond the
same CA env vars introduced in #14178. The change is that more of Codex
now respects them, including websocket-backed flows that were previously
still using default trust roots.
I also manually validated the proxy path locally with mitmproxy using:
`CODEX_CA_CERTIFICATE=~/.mitmproxy/mitmproxy-ca-cert.pem
HTTPS_PROXY=http://127.0.0.1:8080 just codex`
with mitmproxy installed via `brew install mitmproxy` and configured as
the macOS system proxy.
## Mental model
`codex-client` is now the owner of shared custom-CA policy for outbound
TLS client construction. Reqwest callers start from the builder
configuration they already need, then pass that builder through
`build_reqwest_client_with_custom_ca(...)`. Websocket callers ask the
same module for a rustls client config when a custom CA bundle is
configured.
The env precedence is the same everywhere:
- `CODEX_CA_CERTIFICATE` wins
- otherwise fall back to `SSL_CERT_FILE`
- otherwise use system roots
The helper is intentionally narrow. It loads every usable certificate
from the configured PEM bundle into the appropriate root store and
returns either a configured transport or a typed error that explains
what went wrong.
## Non-goals
This does not add handshake-level integration tests against a live TLS
endpoint. It does not validate that the configured bundle forms a
meaningful certificate chain. It also does not try to force every
transport in the repo through one abstraction; it extends the shared CA
policy across the reqwest and websocket paths that actually needed it.
## Tradeoffs
The main tradeoff is centralizing CA behavior in `codex-client` while
still leaving adoption up to call sites. That keeps the implementation
additive and reviewable, but it means the rule "outbound Codex TLS that
should honor enterprise roots must use the shared helper" is still
partly enforced socially rather than by types.
For websockets, the shared helper only builds an explicit rustls config
when a custom CA bundle is configured. When no override env var is set,
websocket callers still use their ordinary default connector path.
## Architecture
`codex-client::custom_ca` now owns CA bundle selection, PEM
normalization, mixed-section parsing, certificate extraction, typed
CA-loading errors, and optional rustls client-config construction for
websocket TLS.
The affected consumers now call into that shared helper directly rather
than carrying login-local CA behavior:
- backend-client
- cloud-tasks
- RMCP client paths that use `reqwest`
- TUI voice HTTP paths
- `codex-core` default reqwest client construction
- `codex-api` websocket clients for both responses and realtime
websocket connections
The subprocess CA probe, env-sensitive integration tests, and shared PEM
fixtures also live in `codex-client`, which is now the actual owner of
the behavior they exercise.
## Observability
The shared CA path logs:
- which environment variable selected the bundle
- which path was loaded
- how many certificates were accepted
- when `TRUSTED CERTIFICATE` labels were normalized
- when CRLs were ignored
- where client construction failed
Returned errors remain user-facing and include the relevant env var,
path, and remediation hint. That same error model now applies whether
the failure surfaced while building a reqwest client or websocket TLS
configuration.
## Tests
Pure unit tests in `codex-client` cover env precedence and PEM
normalization behavior. Real client construction remains in subprocess
tests so the suite can control process env and avoid the macOS seatbelt
panic path that motivated the hermetic test split.
The subprocess coverage verifies:
- `CODEX_CA_CERTIFICATE` precedence over `SSL_CERT_FILE`
- fallback to `SSL_CERT_FILE`
- single-cert and multi-cert bundles
- malformed and empty-file errors
- OpenSSL `TRUSTED CERTIFICATE` handling
- CRL tolerance for well-formed CRL sections
The websocket side is covered by the existing `codex-api` / `codex-core`
websocket test suites plus the manual mitmproxy validation above.
---------
Co-authored-by: Ivan Zakharchanka <3axap4eHko@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-03-12 17:59:26 -07:00
|
|
|
.unwrap_or_else(|error| panic!("failed to run custom_ca_probe: {error}"))
|
login: add custom CA support for login flows (#14178)
## Stacked PRs
This work is split across three stacked PRs:
- #14178: add custom CA support for browser and device-code login flows,
docs, and hermetic subprocess tests
- #14239: broaden the shared custom CA path from login to other outbound
`reqwest` clients across Codex
- #14240: extend that shared custom CA handling to secure websocket TLS
so websocket connections honor the same CA env vars
Review order: #14178, then #14239, then #14240.
Supersedes #6864.
Thanks to @3axap4eHko for the original implementation and investigation
here. Although this version rearranges the code and history
significantly, the majority of the credit for this work belongs to them.
## Problem
Login flows need to work in enterprise environments where outbound TLS
is intercepted by an internal proxy or gateway. In those setups, system
root certificates alone are often insufficient to validate the OAuth and
device-code endpoints used during login. The change adds a
login-specific custom CA loading path, but the important contracts
around env precedence, PEM compatibility, test boundaries, and
probe-only workarounds need to be explicit so reviewers can understand
what behavior is intentional.
For users and operators, the behavior is simple: if login needs to trust
a custom root CA, set `CODEX_CA_CERTIFICATE` to a PEM file containing
one or more certificates. If that variable is unset, login falls back to
`SSL_CERT_FILE`. If neither is set, login uses system roots. Invalid or
empty PEM files now fail with an error that points back to those
environment variables and explains how to recover.
## What This Delivers
Users can now make Codex login work behind enterprise TLS interception
by pointing `CODEX_CA_CERTIFICATE` at a PEM bundle containing the
relevant root certificates. If that variable is unset, login falls back
to `SSL_CERT_FILE`, then to system roots.
This PR applies that behavior to both browser-based and device-code
login flows. It also makes login tolerant of the PEM shapes operators
actually have in hand: multi-certificate bundles, OpenSSL `TRUSTED
CERTIFICATE` labels, and bundles that include well-formed CRLs.
## Mental model
`codex-login` is the place where the login flows construct ad hoc
outbound HTTP clients. That makes it the right boundary for a narrow CA
policy: look for `CODEX_CA_CERTIFICATE`, fall back to `SSL_CERT_FILE`,
load every parseable certificate block in that bundle into a
`reqwest::Client`, and fail early with a clear user-facing error if the
bundle is unreadable or malformed.
The implementation is intentionally pragmatic about PEM input shape. It
accepts ordinary certificate bundles, multi-certificate bundles, OpenSSL
`TRUSTED CERTIFICATE` labels, and bundles that also contain CRLs. It
does not validate a certificate chain or prove a handshake; it only
constructs the root store used by login.
## Non-goals
This change does not introduce a general-purpose transport abstraction
for the rest of the product. It does not validate whether the provided
bundle forms a real chain, and it does not add handshake-level
integration tests against a live TLS server. It also does not change
login state management or OAuth semantics beyond ensuring the existing
flows share the same CA-loading rules.
## Tradeoffs
The main tradeoff is keeping this logic scoped to login-specific client
construction rather than lifting it into a broader shared HTTP layer.
That keeps the review surface smaller, but it also means future
login-adjacent code must continue to use `build_login_http_client()` or
it can silently bypass enterprise CA overrides.
The `TRUSTED CERTIFICATE` handling is also intentionally a local
compatibility shim. The rustls ecosystem does not currently accept that
PEM label upstream, so the code normalizes it locally and trims the
OpenSSL `X509_AUX` trailer bytes down to the certificate DER that
`reqwest` can consume.
## Architecture
`custom_ca.rs` is now the single place that owns login CA behavior. It
selects the CA file from the environment, reads it, normalizes PEM label
shape where needed, iterates mixed PEM sections with `rustls-pki-types`,
ignores CRLs, trims OpenSSL trust metadata when necessary, and returns
either a configured `reqwest::Client` or a typed error.
The browser login server and the device-code flow both call
`build_login_http_client()`, so they share the same trust-store policy.
Environment-sensitive tests run through the `login_ca_probe` helper
binary because those tests must control process-wide env vars and cannot
reliably build a real reqwest client in-process on macOS seatbelt runs.
## Observability
The custom CA path logs which environment variable selected the bundle,
which file path was loaded, how many certificates were accepted, when
`TRUSTED CERTIFICATE` labels were normalized, when CRLs were ignored,
and where client construction failed. Returned errors remain user-facing
and include the relevant path, env var, and remediation hint.
This gives enough signal for three audiences:
- users can see why login failed and which env/file caused it
- sysadmins can confirm which override actually won
- developers can tell whether the failure happened during file read, PEM
parsing, certificate registration, or final reqwest client construction
## Tests
Pure unit tests stay limited to env precedence and empty-value handling.
Real client construction lives in subprocess tests so the suite remains
hermetic with respect to process env and macOS sandbox behavior.
The subprocess tests verify:
- `CODEX_CA_CERTIFICATE` precedence over `SSL_CERT_FILE`
- fallback to `SSL_CERT_FILE`
- single-certificate and multi-certificate bundles
- malformed and empty-bundle errors
- OpenSSL `TRUSTED CERTIFICATE` handling
- CRL tolerance for well-formed CRL sections
The named PEM fixtures under `login/tests/fixtures/` are shared by the
tests so their purpose stays reviewable.
---------
Co-authored-by: Ivan Zakharchanka <3axap4eHko@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-03-12 17:14:54 -07:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
|
fn uses_codex_ca_cert_env() {
|
|
|
|
|
let temp_dir = TempDir::new().expect("tempdir");
|
|
|
|
|
let cert_path = write_cert_file(&temp_dir, "ca.pem", TEST_CERT_1);
|
|
|
|
|
|
|
|
|
|
let output = run_probe(&[(CODEX_CA_CERT_ENV, cert_path.as_path())]);
|
|
|
|
|
|
|
|
|
|
assert!(output.status.success());
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
|
fn falls_back_to_ssl_cert_file() {
|
|
|
|
|
let temp_dir = TempDir::new().expect("tempdir");
|
|
|
|
|
let cert_path = write_cert_file(&temp_dir, "ssl.pem", TEST_CERT_1);
|
|
|
|
|
|
|
|
|
|
let output = run_probe(&[(SSL_CERT_FILE_ENV, cert_path.as_path())]);
|
|
|
|
|
|
|
|
|
|
assert!(output.status.success());
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
|
fn prefers_codex_ca_cert_over_ssl_cert_file() {
|
|
|
|
|
let temp_dir = TempDir::new().expect("tempdir");
|
|
|
|
|
let cert_path = write_cert_file(&temp_dir, "ca.pem", TEST_CERT_1);
|
|
|
|
|
let bad_path = write_cert_file(&temp_dir, "bad.pem", "");
|
|
|
|
|
|
|
|
|
|
let output = run_probe(&[
|
|
|
|
|
(CODEX_CA_CERT_ENV, cert_path.as_path()),
|
|
|
|
|
(SSL_CERT_FILE_ENV, bad_path.as_path()),
|
|
|
|
|
]);
|
|
|
|
|
|
|
|
|
|
assert!(output.status.success());
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
|
fn handles_multi_certificate_bundle() {
|
|
|
|
|
let temp_dir = TempDir::new().expect("tempdir");
|
|
|
|
|
let bundle = format!("{TEST_CERT_1}\n{TEST_CERT_2}");
|
|
|
|
|
let cert_path = write_cert_file(&temp_dir, "bundle.pem", &bundle);
|
|
|
|
|
|
|
|
|
|
let output = run_probe(&[(CODEX_CA_CERT_ENV, cert_path.as_path())]);
|
|
|
|
|
|
|
|
|
|
assert!(output.status.success());
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
|
fn rejects_empty_pem_file_with_hint() {
|
|
|
|
|
let temp_dir = TempDir::new().expect("tempdir");
|
|
|
|
|
let cert_path = write_cert_file(&temp_dir, "empty.pem", "");
|
|
|
|
|
|
|
|
|
|
let output = run_probe(&[(CODEX_CA_CERT_ENV, cert_path.as_path())]);
|
|
|
|
|
|
|
|
|
|
assert!(!output.status.success());
|
|
|
|
|
let stderr = String::from_utf8_lossy(&output.stderr);
|
|
|
|
|
assert!(stderr.contains("no certificates found in PEM file"));
|
|
|
|
|
assert!(stderr.contains("CODEX_CA_CERTIFICATE"));
|
|
|
|
|
assert!(stderr.contains("SSL_CERT_FILE"));
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
|
fn rejects_malformed_pem_with_hint() {
|
|
|
|
|
let temp_dir = TempDir::new().expect("tempdir");
|
|
|
|
|
let cert_path = write_cert_file(
|
|
|
|
|
&temp_dir,
|
|
|
|
|
"malformed.pem",
|
|
|
|
|
"-----BEGIN CERTIFICATE-----\nMIIBroken",
|
|
|
|
|
);
|
|
|
|
|
|
|
|
|
|
let output = run_probe(&[(CODEX_CA_CERT_ENV, cert_path.as_path())]);
|
|
|
|
|
|
|
|
|
|
assert!(!output.status.success());
|
|
|
|
|
let stderr = String::from_utf8_lossy(&output.stderr);
|
|
|
|
|
assert!(stderr.contains("failed to parse PEM file"));
|
|
|
|
|
assert!(stderr.contains("CODEX_CA_CERTIFICATE"));
|
|
|
|
|
assert!(stderr.contains("SSL_CERT_FILE"));
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
|
fn accepts_openssl_trusted_certificate() {
|
|
|
|
|
let temp_dir = TempDir::new().expect("tempdir");
|
|
|
|
|
let cert_path = write_cert_file(&temp_dir, "trusted.pem", TRUSTED_TEST_CERT);
|
|
|
|
|
|
|
|
|
|
let output = run_probe(&[(CODEX_CA_CERT_ENV, cert_path.as_path())]);
|
|
|
|
|
|
|
|
|
|
assert!(output.status.success());
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
|
fn accepts_bundle_with_crl() {
|
|
|
|
|
let temp_dir = TempDir::new().expect("tempdir");
|
|
|
|
|
let crl = "-----BEGIN X509 CRL-----\nMIIC\n-----END X509 CRL-----";
|
|
|
|
|
let bundle = format!("{TEST_CERT_1}\n{crl}");
|
|
|
|
|
let cert_path = write_cert_file(&temp_dir, "bundle_crl.pem", &bundle);
|
|
|
|
|
|
|
|
|
|
let output = run_probe(&[(CODEX_CA_CERT_ENV, cert_path.as_path())]);
|
|
|
|
|
|
|
|
|
|
assert!(output.status.success());
|
|
|
|
|
}
|