aec58ac29b
## Summary - preserve baseline streaming behavior (smooth mode still commits one line per 50ms tick) - extract adaptive chunking policy and commit-tick orchestration from ChatWidget into `streaming/chunking.rs` and `streaming/commit_tick.rs` - add hysteresis-based catch-up behavior with bounded batch draining to reduce queue lag without bursty single-frame jumps - document policy behavior, tuning guidance, and debug flow in rustdoc + docs ## Testing - just fmt - cargo test -p codex-tui
98 lines
3.4 KiB
Markdown
98 lines
3.4 KiB
Markdown
# TUI Stream Chunking Tuning Guide
|
|
|
|
This document explains how to tune adaptive stream chunking constants without
|
|
changing the underlying policy shape.
|
|
|
|
## Scope
|
|
|
|
Use this guide when adjusting queue-pressure thresholds and hysteresis windows in
|
|
`codex-rs/tui/src/streaming/chunking.rs`, and baseline commit cadence in
|
|
`codex-rs/tui/src/app.rs`.
|
|
|
|
This guide is about tuning behavior, not redesigning the policy.
|
|
|
|
## Before tuning
|
|
|
|
- Keep the baseline behavior intact:
|
|
- `Smooth` mode drains one line per baseline tick.
|
|
- `CatchUp` mode drains queued backlog immediately.
|
|
- Capture trace logs with:
|
|
- `codex_tui::streaming::commit_tick`
|
|
- Evaluate on sustained, bursty, and mixed-output prompts.
|
|
|
|
See `docs/tui-stream-chunking-validation.md` for the measurement process.
|
|
|
|
## Tuning goals
|
|
|
|
Tune for all three goals together:
|
|
|
|
- low visible lag under bursty output
|
|
- low mode flapping (`Smooth <-> CatchUp` chatter)
|
|
- stable catch-up entry/exit behavior under mixed workloads
|
|
|
|
## Constants and what they control
|
|
|
|
### Baseline commit cadence
|
|
|
|
- `COMMIT_ANIMATION_TICK` (`tui/src/app.rs`)
|
|
- Lower values increase smooth-mode update cadence and reduce steady-state lag.
|
|
- Higher values increase smoothing and can increase perceived lag.
|
|
- This should usually move after chunking thresholds/holds are in a good range.
|
|
|
|
### Enter/exit thresholds
|
|
|
|
- `ENTER_QUEUE_DEPTH_LINES`, `ENTER_OLDEST_AGE`
|
|
- Lower values enter catch-up earlier (less lag, more mode switching risk).
|
|
- Higher values enter later (more lag tolerance, fewer mode switches).
|
|
- `EXIT_QUEUE_DEPTH_LINES`, `EXIT_OLDEST_AGE`
|
|
- Lower values keep catch-up active longer.
|
|
- Higher values allow earlier exit and may increase re-entry churn.
|
|
|
|
### Hysteresis holds
|
|
|
|
- `EXIT_HOLD`
|
|
- Longer hold reduces flip-flop exits when pressure is noisy.
|
|
- Too long can keep catch-up active after pressure has cleared.
|
|
- `REENTER_CATCH_UP_HOLD`
|
|
- Longer hold suppresses rapid re-entry after exit.
|
|
- Too long can delay needed catch-up for near-term bursts.
|
|
- Severe backlog bypasses this hold by design.
|
|
|
|
### Severe-backlog gates
|
|
|
|
- `SEVERE_QUEUE_DEPTH_LINES`, `SEVERE_OLDEST_AGE`
|
|
- Lower values bypass re-entry hold earlier.
|
|
- Higher values reserve hold bypass for only extreme pressure.
|
|
|
|
## Recommended tuning order
|
|
|
|
Tune in this order to keep cause/effect clear:
|
|
|
|
1. Entry/exit thresholds (`ENTER_*`, `EXIT_*`)
|
|
2. Hold windows (`EXIT_HOLD`, `REENTER_CATCH_UP_HOLD`)
|
|
3. Severe gates (`SEVERE_*`)
|
|
4. Baseline cadence (`COMMIT_ANIMATION_TICK`)
|
|
|
|
Change one logical group at a time and re-measure before the next group.
|
|
|
|
## Symptom-driven adjustments
|
|
|
|
- Too much lag before catch-up starts:
|
|
- lower `ENTER_QUEUE_DEPTH_LINES` and/or `ENTER_OLDEST_AGE`
|
|
- Frequent `Smooth -> CatchUp -> Smooth` chatter:
|
|
- increase `EXIT_HOLD`
|
|
- increase `REENTER_CATCH_UP_HOLD`
|
|
- tighten exit thresholds (lower `EXIT_*`)
|
|
- Catch-up engages too often for short bursts:
|
|
- increase `ENTER_QUEUE_DEPTH_LINES` and/or `ENTER_OLDEST_AGE`
|
|
- increase `REENTER_CATCH_UP_HOLD`
|
|
- Catch-up engages too late:
|
|
- lower `ENTER_QUEUE_DEPTH_LINES` and/or `ENTER_OLDEST_AGE`
|
|
- lower severe gates (`SEVERE_*`) to bypass re-entry hold sooner
|
|
|
|
## Validation checklist after each tuning pass
|
|
|
|
- `cargo test -p codex-tui` passes.
|
|
- Trace window shows bounded queue-age behavior.
|
|
- Mode transitions are not concentrated in repeated short-interval cycles.
|
|
- Catch-up clears backlog quickly once mode enters `CatchUp`.
|