## Summary
This PR makes SQLite rollout backfill resumable and repeatable instead
of one-shot-on-db-create.
## What changed
- Added a persisted backfill state table:
- state/migrations/0008_backfill_state.sql
- Tracks status (pending|running|complete), last_watermark, and
last_success_at.
- Added backfill state model/types in codex-state:
- BackfillState, BackfillStatus (state/src/model/backfill_state.rs)
- Added runtime APIs to manage backfill lifecycle/progress:
- get_backfill_state
- mark_backfill_running
- checkpoint_backfill
- mark_backfill_complete
- Updated core startup behavior:
- Backfill now runs whenever state is not Complete (not only when DB
file is newly created).
- Reworked backfill execution:
- Collect rollout files, derive deterministic watermark per path, sort,
resume from last_watermark.
- Process in batches (BACKFILL_BATCH_SIZE = 200), checkpoint after each
batch.
- Mark complete with last_success_at at the end.
## Why
Previous behavior could leave users permanently partially backfilled if
the process exited during initial async backfill. This change allows
safe continuation across restarts and avoids restarting from scratch.
calculated a hashed user ID from either auth user id or API key
Also correctly populates OS.
These will make our metrics more useful and powerful for analysis.
Summary
- switch the explorer role in core agent configuration to use
`gpt-5.1-codex-mini` as the default model override
- leave other role defaults untouched
Testing
- Not run (not requested)
Summary
- refactor user shell command execution into a shared helper and add
modes for standalone vs active-turn execution
- run user shell commands asynchronously when a turn is already active
so they don’t replace or abort the current turn
- extend the tests to cover the new behavior and add the generated Codex
environment manifest
Testing
- Not run (not requested)
Fixes#10661.
### Problem
On Windows, the sign-in menu can exit immediately if the OS-level input
buffer contains trailing characters (like the Enter key from running the
command).
### Solution
**Flush Input Buffer on Init**: Use FlushConsoleInputBuffer on Windows
(and cflush on Unix) in ui::init() to discard any input captured before
the TUI was ready.
Verified by @CodebyAmbrose in #10661.
## Summary
When resuming with a different model, we should also append a developer
message with the model instructions
## Testing
- [x] Added unit tests
## Summary
This PR fixes a deterministic mismatch in remote compaction where
pre-trim estimation and the `/v1/responses/compact` payload could use
different base instructions.
Before this change:
- pre-trim estimation used model-derived instructions
(`model_info.get_model_instructions(...)`)
- compact payload used session base instructions
(`sess.get_base_instructions()`)
After this change:
- remote pre-trim estimation and compact payload both use the same
`BaseInstructions` instance from session state.
## Changes
- Added a shared estimator entry point in `ContextManager`:
- `estimate_token_count_with_base_instructions(&self, base_instructions:
&BaseInstructions) -> Option<i64>`
- Kept `estimate_token_count(&TurnContext)` as a thin wrapper that
resolves model/personality instructions and delegates to the new helper.
- Updated remote compaction flow to fetch base instructions once and
reuse it for both:
- trim preflight estimation
- compact request payload construction
- Added regression coverage for parity and behavior:
- unit test verifying explicit-base estimator behavior
- integration test proving remote compaction uses session override
instructions and trims accordingly
## Why this matters
This removes a deterministic divergence source where pre-trim could
think the request fits while the actual compact request exceeded context
because its instructions were longer/different.
## Scope
In scope:
- estimator/payload base-instructions parity in remote compaction
Out of scope:
- retry-on-`context_length_exceeded`
- compaction threshold/headroom policy changes
- broader trimming policy changes
## Codex author:
`codex fork 019c2b24-c2df-7b31-a482-fb8cf7a28559`
Promotes the Steer feature from Experimental to Stable and enables it by
default.
## What is Steer mode?
Steer mode changes how message submission works in the TUI:
- **With Steer enabled (new default)**:
- `Enter` submits messages immediately, even when a task is running
- `Tab` queues messages when a task is running (allows building up a
queue)
- **With Steer disabled (old behavior)**:
- `Enter` queues messages when a task is running
- This preserves the previous "queue while a task is running" behavior
## How Steer vs Queue work
The key difference is in the submission behavior:
1. **Steer mode** (`steer_enabled = true`):
- Enter → `InputResult::Submitted` → sends immediately via
`submit_user_message()`
- Tab → `InputResult::Queued` → queues via `queue_user_message()` if a
task is running
- This gives users direct control: Enter for immediate submission, Tab
for queuing
2. **Queue mode** (`steer_enabled = false`, previous default):
- Enter → `InputResult::Queued` → always queues when a task is running
- Tab → `InputResult::Queued` → queues when a task is running
- This preserves the original behavior where Enter respects the running
task queue
## Implementation details
The behavior is controlled in
`ChatComposer::handle_key_event_without_popup()`:
- When `steer_enabled` is true, Enter calls `handle_submission(false)`
(submit immediately)
- When `steer_enabled` is false, Enter calls `handle_submission(true)`
(queue)
See `codex-rs/tui/src/bottom_pane/chat_composer.rs` for the
implementation.
## Documentation
For more details on the chat composer behavior, see:
- [TUI Chat Composer documentation](docs/tui-chat-composer.md)
- Feature flag definition: `codex-rs/core/src/features.rs`
## Summary
- add shared `ModeKind` helpers for display names, TUI visibility, and
`request_user_input` availability
- derive TUI mode filtering/labels from shared `ModeKind` metadata
instead of local hardcoded matches
- derive `request_user_input` availability text and unavailable error
mode names from shared mode metadata
- replace hardcoded known mode names in the Default collaboration-mode
template with `{{KNOWN_MODE_NAMES}}` and fill it from
`TUI_VISIBLE_COLLABORATION_MODES`
- add regression tests for mode metadata sync and placeholder
replacement
## Notes
- `cargo test -p codex-core` integration target (`tests/all`) still
shows pre-existing env-specific failures in this environment due missing
`test_stdio_server` binary resolution; core unit tests are green.
## Codex author
`codex resume 019c26ff-dfe7-7173-bc04-c9e1fff1e447`
## Summary
When switching models, we should append the instructions of the new
model to the conversation as a developer message.
## Test
- [x] Adds a unit test
ensure status indicator present in all classifications of exec tool.
fixes indicator disappearing after preambles, will look into using
`phase` to avoid this class of error in a few hours.
commands parsed as unknown faced this issue
tested locally, added test for specific failure flow
## Problem
When a turn streamed a preamble line before any tool activity,
`ChatWidget` hid the status row while committing streamed lines and did
not restore it until a later event (commonly `ExecCommandBegin`). During
that idle gap, the UI looked finished even though the turn was still
active.
## Mental model
The bottom status row and transcript stream are separate progress
affordances:
- transcript stream shows committed output
- status row (spinner/shimmer + header) shows liveness of an active turn
While stream output is actively committing, hiding the status row is
acceptable to avoid redundant visual noise. Once stream controllers go
idle, an active turn must restore the status row immediately so liveness
remains visible across preamble-to-tool gaps.
## Non-goals
- No changes to streaming chunking policy or pacing.
- No changes to final completion behavior (status still hides when task
actually ends).
- No refactor of status lifecycle ownership between `ChatWidget` and
`BottomPane`.
## Tradeoffs
- We keep the existing behavior of hiding the status row during active
stream commits.
- We add explicit restoration on the idle boundary when the task is
still running.
- This introduces one extra status update on idle transitions, which is
small overhead but makes liveness semantics consistent.
## Architecture
`run_commit_tick_with_scope` in `chatwidget.rs` now documents and
enforces a two-phase contract:
1. For each committed streamed cell, hide status and append transcript
output.
2. If controllers are present and all idle, restore status iff task is
still running, preserving the current header.
This keeps status ownership in `ChatWidget` while relying on
`BottomPane` helpers:
- `hide_status_indicator()` during active stream commits
- `ensure_status_indicator()` +
`set_status_header(current_status_header)` at stream-idle boundary
Documentation pass additions:
- Clarified the function-level contract and lifecycle intent in
`run_commit_tick_with_scope`.
- Added an explicit regression snapshot test comment describing the
failing sequence.
## Observability
Signal that the fix is present:
- In the preamble-idle state, rendered output still includes `• Working
(… esc to interrupt)`.
- New snapshot:
`codex_tui__chatwidget__tests__preamble_keeps_working_status.snap`.
Debug path for future regressions:
- Start at `run_commit_tick_with_scope` for hide/restore transitions.
- Verify `bottom_pane.is_task_running()` at idle transition.
- Confirm `current_status_header` continuity when status is recreated.
- Use the new snapshot and targeted test sequence to reproduce
deterministic preamble-idle behavior.
## Tests
- Updated regression assertion:
- `streaming_final_answer_keeps_task_running_state` now expects status
widget to remain present while turn is running.
- Renamed/updated behavioral regression:
- `preamble_keeps_status_indicator_visible_until_exec_begin`.
- Added snapshot regression coverage:
- `preamble_keeps_working_status_snapshot`.
- Snapshot file:
`tui/src/chatwidget/snapshots/codex_tui__chatwidget__tests__preamble_keeps_working_status.snap`.
Commands run:
- `just fmt`
- `cargo test -p codex-tui
preamble_keeps_status_indicator_visible_until_exec_begin`
- `cargo test -p codex-tui preamble_keeps_working_status_snapshot`
## Risks / Inconsistencies
- Status visibility policy is still split across multiple event paths
(`commit tick`, `turn complete`, `exec begin`), so future regressions
can reintroduce ordering gaps.
- Restoration depends on `is_task_running()` correctness; if task
lifecycle flags drift, status behavior will drift too.
- Snapshot proves rendered state, not animation cadence; cadence still
relies on frame scheduling behavior elsewhere.
Adds a top-level `log_dir` config key (defaults to `$CODEX_HOME/log`) so
one-off runs can redirect `codex-tui.log` via `-c`, e.g.:
codex -c log_dir=./.codex-log
Also resolves relative paths in CLI `-c/--config` overrides for
`AbsolutePathBuf` values against the effective cwd (when available).
Tests:
- cargo test -p codex-core
Make ModelClient a session-scoped object.
Move state that is session level onto the client, and make state that is
per-turn explicit on corresponding methods.
Stop taking a huge Config object, instead only pass in values that are
actually needed.
---------
Co-authored-by: Josh McKinney <joshka@openai.com>
Took over the work that @aaronl-openai started here:
https://github.com/openai/codex/pull/10397
Now that app-server clients are able to set up custom tools (called
`dynamic_tools` in app-server), we should expose a way for clients to
pass in not just text, but also image outputs. This is something the
Responses API already supports for function call outputs, where you can
pass in either a string or an array of content outputs (text, image,
file):
https://platform.openai.com/docs/api-reference/responses/create#responses_create-input-input_item_list-item-function_tool_call_output-output-array-input_image
So let's just plumb it through in Codex (with the caveat that we only
support text and image for now). This is implemented end-to-end across
app-server v2 protocol types and core tool handling.
## Breaking API change
NOTE: This introduces a breaking change with dynamic tools, but I think
it's ok since this concept was only recently introduced
(https://github.com/openai/codex/pull/9539) and it's better to get the
API contract correct. I don't think there are any real consumers of this
yet (not even the Codex App).
Old shape:
`{ "output": "dynamic-ok", "success": true }`
New shape:
```
{
"contentItems": [
{ "type": "inputText", "text": "dynamic-ok" },
{ "type": "inputImage", "imageUrl": "data:image/png;base64,AAA" }
]
"success": true
}
```
Add a centralized FileWatcher in codex-core (using notify) that watches
skill roots from the config layer stack (recursive)
Send `SkillsChanged` events when relevant file system changes are
detected
On `SkillsChanged`:
* Invalidate the skills cache immediately in ThreadManager
* Emit EventMsg::SkillsUpdateAvailable to active sessions
~~* Broadcast a new app-server notification:
SkillsListUpdatedNotification~~
This change does not inject new items into the event stream. That means
the agent will not know about new skills, so it won't be able to
implicitly invoke new skills. It also won't know about changes to
existing skills, so if it has already read the contents of a modified
skill, it will not honor the new behavior.
This change also does not detect modifications to AGENTS.md.
I plan to address these limitations in a follow-on PR modeled after
#9985. Injection of new skills and AGENTS was deemed to risky, hence the
need to split the feature into two stages. The changes in this PR were
designed to easily accommodate the second stage once we have some other
foundational changes in place.
Testing: In addition to automated tests, I did manual testing to confirm
that newly-created skills, deleted skills, and renamed skills are
reflected in the TUI skill picker menu. Also confirmed that
modifications to behaviors for explicitly-invoked skills are honored.
---------
Co-authored-by: Xin Lin <xl@openai.com>
## Human summary
Sandboxing (specifically `LandlockRestrict`) is means that e.g. `sleep
10` fails immediately. Therefore it cannot be interrupted.
In suite::interrupt::test_shell_command_interruption, sleep 10 is issued
at 17:28:16.554 (ToolCall: shell_command {"command":"sleep 10"...}),
then fails at 17:28:16.589 with duration_ms=34, success=false,
exit_code=101, and
Sandbox(LandlockRestrict).
## Codex summary
- set `sandbox_mode = "danger-full-access"` in `interrupt` and
`v2/turn_interrupt` integration tests
- set `sandbox: Some(SandboxMode::DangerFullAccess)` in
`test_codex_jsonrpc_conversation_flow`
- set `sandbox_policy: Some(SandboxPolicy::DangerFullAccess)` in
`command_execution_notifications_include_process_id`
## Why
On some Linux CI environments, command execution fails immediately with
`LandlockRestrict` when sandboxed. These tests are intended to validate
JSON-RPC/task lifecycle behavior (interrupt semantics, command
notification shape/process id, request flow), but early sandbox startup
failure changes turn flow and can trigger extra follow-up requests,
causing flakes.
This change removes environment-specific sandbox startup dependency from
these tests while preserving their primary intent.
## Testing
- not run in this environment (per request)
MCP actions take a long time to load for users with lots of apps
installed. Adding a cache for these actions with 1hr expiration, given
that they are almost always aren't going to change unless people install
another app, which means they also need to restart codex to pick it up.
This PR fixes jitter in the TUI apps menu by making the description
column stable during rendering and height measurement.
Added a `stable_desc_col` option to
`SelectionViewParams`/`ListSelectionView`, introduced stable variants of
the shared row render/measure helpers in `selection_popup_common`, and
enabled the stable mode for the apps/connectors picker in `chatwidget`.
With these changes, only the apps/connectors picker uses this new
option, though it could be used elsewhere in the future.
Why: previously, the description column was computed from only currently
visible rows, so as you scrolled or filtered, the column could shift and
cause wrapping/height changes that looked jumpy. Computing it from all
rows in this popup keeps alignment and layout consistent as users scroll
through avaialble apps.
**Before:**
https://github.com/user-attachments/assets/3856cb72-5465-4b90-a993-65a2ffb09113
**After:**
https://github.com/user-attachments/assets/37b9d626-0b21-4c0f-8bb8-244c9ef971ff
- Schema: thread_id (PK, FK to threads.id with cascade delete),
trace_summary, memory_summary, updated_at.
- Migration: creates the table and an index on (updated_at DESC,
thread_id DESC) for efficient recent-first reads.
- Runtime API (DB-only):
- `get_thread_memory(thread_id)`: fetch one memory row.
- `upsert_thread_memory(thread_id, trace_summary, memory_summary)`:
insert/update by thread id and always advance updated_at.
- `get_last_n_thread_memories_for_cwd(cwd, n)`: join thread_memory with
threads and return newest n rows for an exact cwd match.
- Model layer: introduced ThreadMemory and row conversion types to keep
query decoding typed and consistent with existing state models.
## Summary
This PR introduces a gated Bubblewrap (bwrap) Linux sandbox path. The
curent Linux sandbox path relies on in-process restrictions (including
Landlock). Bubblewrap gives us a more uniform filesystem isolation
model, especially explicit writable roots with the option to make some
directories read-only and granular network controls.
This is behind a feature flag so we can validate behavior safely before
making it the default.
- Added temporary rollout flag:
- `features.use_linux_sandbox_bwrap`
- Preserved existing default path when the flag is off.
- In Bubblewrap mode:
- Added internal retry without /proc when /proc mount is not permitted
by the host/container.
This PR adds a new approval option for app/MCP tool calls: “Allow and
remember” (session-scoped).
When selected, Codex stores a temporary approval and auto-approves
matching future calls for the rest of the session.
Added a session-scoped approval key (`server`, `connector_id`,
`tool_name`) and persisted it in `tool_approvals` as
`ApprovedForSession`.
On subsequent matching calls, approval is skipped and treated as
accepted.
- Updated the approval question options to conditionally include:
- Accept
- Allow and remember (conditional)
- Decline
- Cancel
The new “Allow and remember” option is only shown when all of these are
true:
1. The call is routed through the Codex Apps MCP server (codex_apps).
2. The tool requires approval based on annotations:
- read_only_hint == false, and
- destructive_hint == true or open_world_hint == true.
3. The tool includes a connector_id in metadata (used to build the
remembered approval key).
If no `connector_id` is present, the prompt still appears (when approval
is required), but only with the existing choices (Accept / Decline /
Cancel). Approval prompting in this path has an explicit early return
unless server == `codex_apps`.
Summary:
- replace the `sse_completed` fixture and related JSON template with
direct `responses::ev_completed` payload builders
- cascade the new SSE helpers through all affected core tests for
consistency and clarity
- remove legacy fixtures that were no longer needed once the helpers are
in place
Testing:
- Not run (not requested)
Summary
- add versioned state sqlite filename helpers and re-export them from
the state crate
- remove legacy state files when initializing the runtime and update
consumers/tests to use the new helpers
- tweak logs client description and database resolution to match the new
path
When communicating over websockets, we can't rely on headers to deliver
rate limit information. This PR adds a `codex.rate_limits` event that
the server can pass to the client to inform them about rate limit usage.
The client parses this data the same way we parse rate limit headers in
HTTP mode.
This PR also wires up the etag and reasoning headers for websockets
If we want to build `/debug-config`, we'll need to know the requirements
sources that supplied the values.
This PR adds those sources such that we can render them in the UI.
