Commit graph

17 commits

Author SHA1 Message Date
jif-oai
3d8deeea4b
fix: single transaction for dyn tools injection (#10614) 2026-02-04 10:57:58 +00:00
jif-oai
100eb6e6f0
Prefer state DB thread listings before filesystem (#10544)
Summary
- add Cursor/ThreadsPage conversions so state DB listings can be mapped
back into the rollout list model
- make recorder list helpers query the state DB first (archived flag
included) and only fall back to file traversal if needed, along with
populating head bytes lazily
- add extensive tests to ensure the DB path is honored for active and
archived threads and that the fallback works

Testing
- Not run (not requested)

<img width="1196" height="693" alt="Screenshot 2026-02-03 at 20 42 33"
src="https://github.com/user-attachments/assets/826b3c7a-ef11-4b27-802a-3c343695794a"
/>
2026-02-04 09:27:24 +00:00
jif-oai
ed778f9017
Avoid redundant transactional check before inserting dynamic tools (#10521)
Summary
- remove the extra transaction guard that checked for existing dynamic
tools per thread before inserting new ones
- insert each tool record with `ON CONFLICT(thread_id, position) DO
NOTHING` to ignore duplicates instead of pre-querying
- simplify execution to use the shared pool directly and avoid unneeded
commits

Testing
- Not run (not requested)
2026-02-03 15:34:28 +00:00
sayan-oai
fc05374344
chore: add phase to message responseitem (#10455)
### What

add wiring for `phase` field on `ResponseItem::Message` to lay
groundwork for differentiating model preambles and final messages.
currently optional.

follows pattern in #9698.

updated schemas with `just write-app-server-schema` so we can see type
changes.

### Tests
Updated existing tests for SSE parsing and hydrating from history
2026-02-03 02:52:26 +00:00
Celia Chen
fb2df99cf1
[feat] persist thread_dynamic_tools in db (#10252)
Persist thread_dynamic_tools in sqlite and read first from it. Fall back
to rollout files if it's not found. Persist dynamic tools to both sqlite
and rollout files.

Saw that new sessions get populated to db correctly & old sessions get
backfilled correctly at startup:
```
celia@com-92114 codex-rs % sqlite3 ~/.codex/state.sqlite \      "select thread_id, position,name,description,input_schema from thread_dynamic_tools;"
019c0cad-ec0d-74b2-a787-e8b33a349117|0|geo_lookup|lookup a city|{"properties":{"city":{"type":"string"}},"required":["city"],"type":"object"}
....
019c10ca-aa4b-7620-ae40-c0919fbd7ea7|0|geo_lookup|lookup a city|{"properties":{"city":{"type":"string"}},"required":["city"],"type":"object"}
```
2026-02-03 00:06:44 +00:00
jif-oai
34c0534f6e
feat: drop sqlx logging (#10398) 2026-02-02 19:26:58 +00:00
jif-oai
0212f4010e
nit: fix db with multiple metadata lines (#10237) 2026-01-30 17:32:10 +00:00
jif-oai
eff11f792b
feat: improve logs client (#10229) 2026-01-30 18:23:18 +01:00
jif-oai
129787493f
feat: backfill timing metric (#10218)
1. Add a metric to measure the backfill time
2. Add a unit to the timing histogram
2026-01-30 10:19:41 +01:00
jif-oai
798c4b3260
feat: reduce span exposition (#10171)
This only avoids the creation of duplicates spans
2026-01-29 18:15:22 +00:00
jif-oai
e6c4f548ab
chore: unify log queries (#10152)
Unify log queries to only have SQLX code in the runtime and use it for
both the log client and for tests
2026-01-29 16:28:15 +00:00
jif-oai
d6631fb5a9
feat: add log retention and delete them after 90 days (#10151) 2026-01-29 16:55:01 +01:00
jif-oai
89c5f3c4d4
feat: adding thread ID to logs + filter in the client (#10150) 2026-01-29 16:53:30 +01:00
jif-oai
4ba911d48c
chore: improve client (#10149)
<img width="883" height="84" alt="Screenshot 2026-01-29 at 11 13 12"
src="https://github.com/user-attachments/assets/090a2fec-94ed-4c0f-aee5-1653ed8b1439"
/>
2026-01-29 11:25:22 +01:00
jif-oai
6a06726af2
feat: log db client (#10087)
```
just log -h
if [ "${1:-}" = "--" ]; then shift; fi; cargo run -p codex-state --bin logs_client -- "$@"
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.21s
     Running `target/debug/logs_client -h`
Tail Codex logs from state.sqlite with simple filters

Usage: logs_client [OPTIONS]

Options:
      --codex-home <CODEX_HOME>  Path to CODEX_HOME. Defaults to $CODEX_HOME or ~/.codex [env: CODEX_HOME=]
      --db <DB>                  Direct path to the SQLite database. Overrides --codex-home
      --level <LEVEL>            Log level to match exactly (case-insensitive)
      --from <RFC3339|UNIX>      Start timestamp (RFC3339 or unix seconds)
      --to <RFC3339|UNIX>        End timestamp (RFC3339 or unix seconds)
      --module <MODULE>          Substring match on module_path
      --file <FILE>              Substring match on file path
      --backfill <BACKFILL>      Number of matching rows to show before tailing [default: 200]
      --poll-ms <POLL_MS>        Poll interval in milliseconds [default: 500]
  -h, --help                     Print help
  ```
2026-01-29 11:11:47 +01:00
jif-oai
780482da84
feat: add log db (#10086)
Add a log DB. The goal is just to store our logs in a `.sqlite` DB to
make it easier to crawl them and drop the oldest ones.
2026-01-29 10:23:03 +01:00
jif-oai
3878c3dc7c
feat: sqlite 1 (#10004)
Add a `.sqlite` database to be used to store rollout metatdata (and
later logs)
This PR is phase 1:
* Add the database and the required infrastructure
* Add a backfill of the database
* Persist the newly created rollout both in files and in the DB
* When we need to get metadata or a rollout, consider the `JSONL` as the
source of truth but compare the results with the DB and show any errors
2026-01-28 15:29:14 +01:00