c4f1af7a86
## Summary Adds syntax highlighting to the TUI for fenced code blocks in markdown responses and file diffs, plus a `/theme` command with live preview and persistent theme selection. Uses syntect (~250 grammars, 32 bundled themes, ~1 MB binary cost) — the same engine behind `bat`, `delta`, and `xi-editor`. Includes guardrails for large inputs, graceful fallback to plain text, and SSH-aware clipboard integration for the `/copy` command. <img width="1554" height="1014" alt="image" src="https://github.com/user-attachments/assets/38737a79-8717-4715-b857-94cf1ba59b85" /> <img width="2354" height="1374" alt="image" src="https://github.com/user-attachments/assets/25d30a00-c487-4af8-9cb6-63b0695a4be7" /> ## Problem Code blocks in the TUI (markdown responses and file diffs) render without syntax highlighting, making it hard to scan code at a glance. Users also have no way to pick a color theme that matches their terminal aesthetic. ## Mental model The highlighting system has three layers: 1. **Syntax engine** (`render::highlight`) -- a thin wrapper around syntect + two-face. It owns a process-global `SyntaxSet` (~250 grammars) and a `RwLock<Theme>` that can be swapped at runtime. All public entry points accept `(code, lang)` and return ratatui `Span`/`Line` vectors or `None` when the language is unrecognized or the input exceeds safety guardrails. 2. **Rendering consumers** -- `markdown_render` feeds fenced code blocks through the engine; `diff_render` highlights Add/Delete content as a whole file and Update hunks per-hunk (preserving parser state across hunk lines). Both callers fall back to plain unstyled text when the engine returns `None`. 3. **Theme lifecycle** -- at startup the config's `tui.theme` is resolved to a syntect `Theme` via `set_theme_override`. At runtime the `/theme` picker calls `set_syntax_theme` to swap themes live; on cancel it restores the snapshot taken at open. On confirm it persists `[tui] theme = "..."` to config.toml. ## Non-goals - Inline diff highlighting (word-level change detection within a line). - Semantic / LSP-backed highlighting. - Theme authoring tooling; users supply standard `.tmTheme` files. ## Tradeoffs | Decision | Upside | Downside | | ------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | syntect over tree-sitter / arborium | ~1 MB binary increase for ~250 grammars + 32 themes; battle-tested crate powering widely-used tools (`bat`, `delta`, `xi-editor`). tree-sitter would add ~12 MB for 20-30 languages or ~35 MB for full coverage. | Regex-based; less structurally accurate than tree-sitter for some languages (e.g. language injections like JS-in-HTML). | | Global `RwLock<Theme>` | Enables live `/theme` preview without threading Theme through every call site | Lock contention risk (mitigated: reads vastly outnumber writes, single UI thread) | | Skip background / italic / underline from themes | Terminal BG preserved, avoids ugly rendering on some themes | Themes that rely on these properties lose fidelity | | Guardrails: 512 KB / 10k lines | Prevents pathological stalls on huge diffs or pastes | Very large files render without color | ## Architecture ``` config.toml ─[tui.theme]─> set_theme_override() ─> THEME (RwLock) │ ┌───────────────────────────────────────────┘ │ markdown_render ─── highlight_code_to_lines(code, lang) ─> Vec<Line> diff_render ─── highlight_code_to_styled_spans(code, lang) ─> Option<Vec<Vec<Span>>> │ │ (None ⇒ plain text fallback) │ /theme picker ─── set_syntax_theme(theme) // live preview swap ─── current_syntax_theme() // snapshot for cancel ─── resolve_theme_by_name(name) // lookup by kebab-case ``` Key files: - `tui/src/render/highlight.rs` -- engine, theme management, guardrails - `tui/src/diff_render.rs` -- syntax-aware diff line wrapping - `tui/src/theme_picker.rs` -- `/theme` command builder - `tui/src/bottom_pane/list_selection_view.rs` -- side content panel, callbacks - `core/src/config/types.rs` -- `Tui::theme` field - `core/src/config/edit.rs` -- `syntax_theme_edit()` helper ## Observability - `tracing::warn` when a configured theme name cannot be resolved. - `Config::startup_warnings` surfaces the same message as a TUI banner. - `tracing::error` when persisting theme selection fails. ## Tests - Unit tests in `highlight.rs`: language coverage, fallback behavior, CRLF stripping, style conversion, guardrail enforcement, theme name mapping exhaustiveness. - Unit tests in `diff_render.rs`: snapshot gallery at multiple terminal sizes (80x24, 94x35, 120x40), syntax-highlighted wrapping, large-diff guardrail, rename-to-different-extension highlighting, parser state preservation across hunk lines. - Unit tests in `theme_picker.rs`: preview rendering (wide + narrow), dim overlay on deletions, subtitle truncation, cancel-restore, fallback for unavailable configured theme. - Unit tests in `list_selection_view.rs`: side layout geometry, stacked fallback, buffer clearing, cancel/selection-changed callbacks. - Integration test in `lib.rs`: theme warning uses the final (post-resume) config. ## Cargo Deny: Unmaintained Dependency Exceptions This PR adds two `cargo deny` advisory exceptions for transitive dependencies pulled in by `syntect v5.3.0`: | Advisory | Crate | Status | |----------|-------|--------| | RUSTSEC-2024-0320 | `yaml-rust` | Unmaintained (maintainer unreachable) | | RUSTSEC-2025-0141 | `bincode` | Unmaintained (development ceased; v1.3.3 considered complete) | **Why this is safe in our usage:** - Neither advisory describes a known security vulnerability. Both are "unmaintained" notices only. - `bincode` is used by syntect to deserialize pre-compiled syntax sets. Again, these are **static vendored artifacts** baked into the binary at build time. No user-supplied bincode data is ever deserialized. - Attack surface is zero for both crates; exploitation would require a supply-chain compromise of our own build artifacts. - These exceptions can be removed when syntect migrates to `yaml-rust2` and drops `bincode`, or when alternative crates are available upstream.
373 lines
10 KiB
TOML
373 lines
10 KiB
TOML
[workspace]
|
|
members = [
|
|
"backend-client",
|
|
"ansi-escape",
|
|
"async-utils",
|
|
"app-server",
|
|
"app-server-protocol",
|
|
"app-server-test-client",
|
|
"debug-client",
|
|
"apply-patch",
|
|
"arg0",
|
|
"feedback",
|
|
"codex-backend-openapi-models",
|
|
"cloud-requirements",
|
|
"cloud-tasks",
|
|
"cloud-tasks-client",
|
|
"cli",
|
|
"config",
|
|
"shell-command",
|
|
"skills",
|
|
"core",
|
|
"hooks",
|
|
"secrets",
|
|
"exec",
|
|
"exec-server",
|
|
"execpolicy",
|
|
"execpolicy-legacy",
|
|
"keyring-store",
|
|
"file-search",
|
|
"linux-sandbox",
|
|
"lmstudio",
|
|
"login",
|
|
"mcp-server",
|
|
"network-proxy",
|
|
"ollama",
|
|
"process-hardening",
|
|
"protocol",
|
|
"rmcp-client",
|
|
"responses-api-proxy",
|
|
"stdio-to-uds",
|
|
"otel",
|
|
"tui",
|
|
"utils/absolute-path",
|
|
"utils/cargo-bin",
|
|
"utils/git",
|
|
"utils/cache",
|
|
"utils/image",
|
|
"utils/json-to-toml",
|
|
"utils/home-dir",
|
|
"utils/pty",
|
|
"utils/readiness",
|
|
"utils/rustls-provider",
|
|
"utils/string",
|
|
"utils/cli",
|
|
"utils/elapsed",
|
|
"utils/sandbox-summary",
|
|
"utils/sleep-inhibitor",
|
|
"utils/approval-presets",
|
|
"utils/oss",
|
|
"utils/fuzzy-match",
|
|
"codex-client",
|
|
"codex-api",
|
|
"state",
|
|
"codex-experimental-api-macros",
|
|
]
|
|
resolver = "2"
|
|
|
|
[workspace.package]
|
|
version = "0.0.0"
|
|
# Track the edition for all workspace crates in one place. Individual
|
|
# crates can still override this value, but keeping it here means new
|
|
# crates created with `cargo new -w ...` automatically inherit the 2024
|
|
# edition.
|
|
edition = "2024"
|
|
license = "Apache-2.0"
|
|
|
|
[workspace.dependencies]
|
|
# Internal
|
|
app_test_support = { path = "app-server/tests/common" }
|
|
codex-ansi-escape = { path = "ansi-escape" }
|
|
codex-api = { path = "codex-api" }
|
|
codex-app-server = { path = "app-server" }
|
|
codex-app-server-protocol = { path = "app-server-protocol" }
|
|
codex-app-server-test-client = { path = "app-server-test-client" }
|
|
codex-apply-patch = { path = "apply-patch" }
|
|
codex-arg0 = { path = "arg0" }
|
|
codex-async-utils = { path = "async-utils" }
|
|
codex-backend-client = { path = "backend-client" }
|
|
codex-chatgpt = { path = "chatgpt" }
|
|
codex-cli = { path = "cli" }
|
|
codex-client = { path = "codex-client" }
|
|
codex-cloud-requirements = { path = "cloud-requirements" }
|
|
codex-config = { path = "config" }
|
|
codex-core = { path = "core" }
|
|
codex-exec = { path = "exec" }
|
|
codex-execpolicy = { path = "execpolicy" }
|
|
codex-experimental-api-macros = { path = "codex-experimental-api-macros" }
|
|
codex-feedback = { path = "feedback" }
|
|
codex-file-search = { path = "file-search" }
|
|
codex-git = { path = "utils/git" }
|
|
codex-hooks = { path = "hooks" }
|
|
codex-keyring-store = { path = "keyring-store" }
|
|
codex-linux-sandbox = { path = "linux-sandbox" }
|
|
codex-lmstudio = { path = "lmstudio" }
|
|
codex-login = { path = "login" }
|
|
codex-mcp-server = { path = "mcp-server" }
|
|
codex-network-proxy = { path = "network-proxy" }
|
|
codex-ollama = { path = "ollama" }
|
|
codex-otel = { path = "otel" }
|
|
codex-process-hardening = { path = "process-hardening" }
|
|
codex-protocol = { path = "protocol" }
|
|
codex-responses-api-proxy = { path = "responses-api-proxy" }
|
|
codex-rmcp-client = { path = "rmcp-client" }
|
|
codex-secrets = { path = "secrets" }
|
|
codex-shell-command = { path = "shell-command" }
|
|
codex-skills = { path = "skills" }
|
|
codex-state = { path = "state" }
|
|
codex-stdio-to-uds = { path = "stdio-to-uds" }
|
|
codex-tui = { path = "tui" }
|
|
codex-utils-absolute-path = { path = "utils/absolute-path" }
|
|
codex-utils-approval-presets = { path = "utils/approval-presets" }
|
|
codex-utils-cache = { path = "utils/cache" }
|
|
codex-utils-cargo-bin = { path = "utils/cargo-bin" }
|
|
codex-utils-cli = { path = "utils/cli" }
|
|
codex-utils-elapsed = { path = "utils/elapsed" }
|
|
codex-utils-fuzzy-match = { path = "utils/fuzzy-match" }
|
|
codex-utils-home-dir = { path = "utils/home-dir" }
|
|
codex-utils-image = { path = "utils/image" }
|
|
codex-utils-json-to-toml = { path = "utils/json-to-toml" }
|
|
codex-utils-oss = { path = "utils/oss" }
|
|
codex-utils-pty = { path = "utils/pty" }
|
|
codex-utils-readiness = { path = "utils/readiness" }
|
|
codex-utils-rustls-provider = { path = "utils/rustls-provider" }
|
|
codex-utils-sandbox-summary = { path = "utils/sandbox-summary" }
|
|
codex-utils-sleep-inhibitor = { path = "utils/sleep-inhibitor" }
|
|
codex-utils-string = { path = "utils/string" }
|
|
codex-windows-sandbox = { path = "windows-sandbox-rs" }
|
|
core_test_support = { path = "core/tests/common" }
|
|
exec_server_test_support = { path = "exec-server/tests/common" }
|
|
mcp_test_support = { path = "mcp-server/tests/common" }
|
|
|
|
# External
|
|
age = "0.11.1"
|
|
allocative = "0.3.3"
|
|
ansi-to-tui = "7.0.0"
|
|
anyhow = "1"
|
|
arboard = { version = "3", features = ["wayland-data-control"] }
|
|
assert_cmd = "2"
|
|
assert_matches = "1.5.0"
|
|
async-channel = "2.3.1"
|
|
async-stream = "0.3.6"
|
|
async-trait = "0.1.89"
|
|
axum = { version = "0.8", default-features = false }
|
|
base64 = "0.22.1"
|
|
bm25 = "2.3.2"
|
|
bytes = "1.10.1"
|
|
chardetng = "0.1.17"
|
|
chrono = "0.4.43"
|
|
clap = "4"
|
|
clap_complete = "4"
|
|
color-eyre = "0.6.3"
|
|
crossbeam-channel = "0.5.15"
|
|
crossterm = "0.28.1"
|
|
ctor = "0.6.3"
|
|
derive_more = "2"
|
|
diffy = "0.4.2"
|
|
dirs = "6"
|
|
dotenvy = "0.15.7"
|
|
dunce = "1.0.4"
|
|
encoding_rs = "0.8.35"
|
|
env-flags = "0.1.1"
|
|
env_logger = "0.11.9"
|
|
eventsource-stream = "0.2.3"
|
|
futures = { version = "0.3", default-features = false }
|
|
globset = "0.4"
|
|
http = "1.3.1"
|
|
icu_decimal = "2.1"
|
|
icu_locale_core = "2.1"
|
|
icu_provider = { version = "2.1", features = ["sync"] }
|
|
ignore = "0.4.23"
|
|
image = { version = "^0.25.9", default-features = false }
|
|
include_dir = "0.7.4"
|
|
indexmap = "2.12.0"
|
|
indoc = "2.0"
|
|
insta = "1.46.3"
|
|
inventory = "0.3.19"
|
|
itertools = "0.14.0"
|
|
keyring = { version = "3.6", default-features = false }
|
|
landlock = "0.4.4"
|
|
lazy_static = "1"
|
|
libc = "0.2.177"
|
|
log = "0.4"
|
|
lru = "0.16.3"
|
|
maplit = "1.0.2"
|
|
mime_guess = "2.0.5"
|
|
multimap = "0.10.0"
|
|
notify = "8.2.0"
|
|
nucleo = { git = "https://github.com/helix-editor/nucleo.git", rev = "4253de9faabb4e5c6d81d946a5e35a90f87347ee" }
|
|
once_cell = "1.20.2"
|
|
openssl-sys = "*"
|
|
opentelemetry = "0.31.0"
|
|
opentelemetry-appender-tracing = "0.31.0"
|
|
opentelemetry-otlp = "0.31.0"
|
|
opentelemetry-semantic-conventions = "0.31.0"
|
|
opentelemetry_sdk = "0.31.0"
|
|
os_info = "3.12.0"
|
|
owo-colors = "4.2.0"
|
|
path-absolutize = "3.1.1"
|
|
pathdiff = "0.2"
|
|
portable-pty = "0.9.0"
|
|
predicates = "3"
|
|
pretty_assertions = "1.4.1"
|
|
pulldown-cmark = "0.10"
|
|
rand = "0.9"
|
|
ratatui = "0.29.0"
|
|
ratatui-macros = "0.6.0"
|
|
regex = "1.12.3"
|
|
regex-lite = "0.1.8"
|
|
reqwest = "0.12"
|
|
rmcp = { version = "0.15.0", default-features = false }
|
|
runfiles = { git = "https://github.com/dzbarsky/rules_rust", rev = "b56cbaa8465e74127f1ea216f813cd377295ad81" }
|
|
rustls = { version = "0.23", default-features = false, features = [
|
|
"ring",
|
|
"std",
|
|
] }
|
|
schemars = "0.8.22"
|
|
seccompiler = "0.5.0"
|
|
semver = "1.0"
|
|
sentry = "0.46.0"
|
|
serde = "1"
|
|
serde_json = "1"
|
|
serde_path_to_error = "0.1.20"
|
|
serde_with = "3.16"
|
|
serde_yaml = "0.9"
|
|
serial_test = "3.2.0"
|
|
sha1 = "0.10.6"
|
|
sha2 = "0.10"
|
|
shlex = "1.3.0"
|
|
similar = "2.7.0"
|
|
socket2 = "0.6.1"
|
|
sqlx = { version = "0.8.6", default-features = false, features = [
|
|
"chrono",
|
|
"json",
|
|
"macros",
|
|
"migrate",
|
|
"runtime-tokio-rustls",
|
|
"sqlite",
|
|
"time",
|
|
"uuid",
|
|
] }
|
|
starlark = "0.13.0"
|
|
strum = "0.27.2"
|
|
strum_macros = "0.27.2"
|
|
supports-color = "3.0.2"
|
|
sys-locale = "0.3.2"
|
|
tempfile = "3.23.0"
|
|
test-log = "0.2.19"
|
|
textwrap = "0.16.2"
|
|
thiserror = "2.0.17"
|
|
time = "0.3.47"
|
|
tiny_http = "0.12"
|
|
tokio = "1"
|
|
tokio-stream = "0.1.18"
|
|
tokio-test = "0.4"
|
|
tokio-tungstenite = { version = "0.28.0", features = [
|
|
"proxy",
|
|
"rustls-tls-native-roots",
|
|
] }
|
|
tokio-util = "0.7.18"
|
|
toml = "0.9.5"
|
|
toml_edit = "0.24.0"
|
|
tracing = "0.1.44"
|
|
tracing-appender = "0.2.3"
|
|
tracing-opentelemetry = "0.32.0"
|
|
tracing-subscriber = "0.3.22"
|
|
tracing-test = "0.2.5"
|
|
tree-sitter = "0.25.10"
|
|
tree-sitter-bash = "0.25"
|
|
syntect = "5"
|
|
ts-rs = "11"
|
|
tungstenite = { version = "0.27.0", features = ["deflate", "proxy"] }
|
|
uds_windows = "1.1.0"
|
|
unicode-segmentation = "1.12.0"
|
|
unicode-width = "0.2"
|
|
url = "2"
|
|
urlencoding = "2.1"
|
|
uuid = "1"
|
|
vt100 = "0.16.2"
|
|
walkdir = "2.5.0"
|
|
webbrowser = "1.0"
|
|
which = "8"
|
|
wildmatch = "2.6.1"
|
|
zip = "2.4.2"
|
|
zstd = "0.13"
|
|
|
|
wiremock = "0.6"
|
|
zeroize = "1.8.2"
|
|
|
|
[workspace.lints]
|
|
rust = {}
|
|
|
|
[workspace.lints.clippy]
|
|
expect_used = "deny"
|
|
identity_op = "deny"
|
|
manual_clamp = "deny"
|
|
manual_filter = "deny"
|
|
manual_find = "deny"
|
|
manual_flatten = "deny"
|
|
manual_map = "deny"
|
|
manual_memcpy = "deny"
|
|
manual_non_exhaustive = "deny"
|
|
manual_ok_or = "deny"
|
|
manual_range_contains = "deny"
|
|
manual_retain = "deny"
|
|
manual_strip = "deny"
|
|
manual_try_fold = "deny"
|
|
manual_unwrap_or = "deny"
|
|
needless_borrow = "deny"
|
|
needless_borrowed_reference = "deny"
|
|
needless_collect = "deny"
|
|
needless_late_init = "deny"
|
|
needless_option_as_deref = "deny"
|
|
needless_question_mark = "deny"
|
|
needless_update = "deny"
|
|
redundant_clone = "deny"
|
|
redundant_closure = "deny"
|
|
redundant_closure_for_method_calls = "deny"
|
|
redundant_static_lifetimes = "deny"
|
|
trivially_copy_pass_by_ref = "deny"
|
|
uninlined_format_args = "deny"
|
|
unnecessary_filter_map = "deny"
|
|
unnecessary_lazy_evaluations = "deny"
|
|
unnecessary_sort_by = "deny"
|
|
unnecessary_to_owned = "deny"
|
|
unwrap_used = "deny"
|
|
|
|
# cargo-shear cannot see the platform-specific openssl-sys usage, so we
|
|
# silence the false positive here instead of deleting a real dependency.
|
|
[workspace.metadata.cargo-shear]
|
|
ignored = [
|
|
"icu_provider",
|
|
"openssl-sys",
|
|
"codex-utils-readiness",
|
|
"codex-secrets",
|
|
]
|
|
|
|
[profile.release]
|
|
lto = "fat"
|
|
# Because we bundle some of these executables with the TypeScript CLI, we
|
|
# remove everything to make the binary as small as possible.
|
|
strip = "symbols"
|
|
|
|
# See https://github.com/openai/codex/issues/1411 for details.
|
|
codegen-units = 1
|
|
|
|
[profile.ci-test]
|
|
debug = 1 # Reduce debug symbol size
|
|
inherits = "test"
|
|
opt-level = 0
|
|
|
|
[patch.crates-io]
|
|
# Uncomment to debug local changes.
|
|
# ratatui = { path = "../../ratatui" }
|
|
crossterm = { git = "https://github.com/nornagon/crossterm", branch = "nornagon/color-query" }
|
|
ratatui = { git = "https://github.com/nornagon/ratatui", branch = "nornagon-v0.29.0-patch" }
|
|
tokio-tungstenite = { git = "https://github.com/openai-oss-forks/tokio-tungstenite", rev = "132f5b39c862e3a970f731d709608b3e6276d5f6" }
|
|
tungstenite = { git = "https://github.com/openai-oss-forks/tungstenite-rs", rev = "9200079d3b54a1ff51072e24d81fd354f085156f" }
|
|
|
|
# Uncomment to debug local changes.
|
|
# rmcp = { path = "../../rust-sdk/crates/rmcp" }
|
|
|
|
[patch."ssh://git@github.com/openai-oss-forks/tungstenite-rs.git"]
|
|
tungstenite = { git = "https://github.com/openai-oss-forks/tungstenite-rs", rev = "9200079d3b54a1ff51072e24d81fd354f085156f" }
|