core-agent-ide/codex-rs/app-server-protocol/schema/json
Curtis 'Fjord' Hawthorne 7e980d7db6
Support multimodal custom tool outputs (#12948)
## Summary

This changes `custom_tool_call_output` to use the same output payload
shape as `function_call_output`, so freeform tools can return either
plain text or structured content items.

The main goal is to let `js_repl` return image content from nested
`view_image` calls in its own `custom_tool_call_output`, instead of
relying on a separate injected message.

## What changed

- Changed `custom_tool_call_output.output` from `string` to
`FunctionCallOutputPayload`
- Updated freeform tool plumbing to preserve structured output bodies
- Updated `js_repl` to aggregate nested tool content items and attach
them to the outer `js_repl` result
- Removed the old `js_repl` special case that injected `view_image`
results as a separate pending user image message
- Updated normalization/history/truncation paths to handle multimodal
`custom_tool_call_output`
- Regenerated app-server protocol schema artifacts

## Behavior

Direct `view_image` calls still return a `function_call_output` with
image content.

When `view_image` is called inside `js_repl`, the outer `js_repl`
`custom_tool_call_output` now carries:
- an `input_text` item if the JS produced text output
- one or more `input_image` items from nested tool results

So the nested image result now stays inside the `js_repl` tool output
instead of being injected as a separate message.

## Compatibility

This is intended to be backward-compatible for resumed conversations.

Older histories that stored `custom_tool_call_output.output` as a plain
string still deserialize correctly, and older histories that used the
previous injected-image-message flow also continue to resume.

Added regression coverage for resuming a pre-change rollout containing:
- string-valued `custom_tool_call_output`
- legacy injected image message history


#### [git stack](https://github.com/magus/git-stack-cli)
- 👉 `1` https://github.com/openai/codex/pull/12948
2026-02-26 18:17:46 -08:00
..
v1 ignore v1 in JSON schema codegen (#12408) 2026-02-20 21:36:12 -08:00
v2 Support multimodal custom tool outputs (#12948) 2026-02-26 18:17:46 -08:00
ApplyPatchApprovalParams.json chore: remove codex-core public protocol/shell re-exports (#12432) 2026-02-20 23:45:35 -08:00
ApplyPatchApprovalResponse.json feat: include available decisions in command approval requests (#12758) 2026-02-26 01:10:46 +00:00
ChatgptAuthTokensRefreshParams.json fix(app-server): for external auth, replace id_token with chatgpt_acc… (#11240) 2026-02-09 20:48:58 -08:00
ChatgptAuthTokensRefreshResponse.json fix(app-server): for external auth, replace id_token with chatgpt_acc… (#11240) 2026-02-09 20:48:58 -08:00
ClientNotification.json feat: vendor app-server protocol schema fixtures (#10371) 2026-02-01 23:38:43 -08:00
ClientRequest.json Support multimodal custom tool outputs (#12948) 2026-02-26 18:17:46 -08:00
codex_app_server_protocol.schemas.json Support multimodal custom tool outputs (#12948) 2026-02-26 18:17:46 -08:00
CommandExecutionRequestApprovalParams.json feat: include available decisions in command approval requests (#12758) 2026-02-26 01:10:46 +00:00
CommandExecutionRequestApprovalResponse.json feat: include available decisions in command approval requests (#12758) 2026-02-26 01:10:46 +00:00
DynamicToolCallParams.json feat: vendor app-server protocol schema fixtures (#10371) 2026-02-01 23:38:43 -08:00
DynamicToolCallResponse.json feat(app-server, core): allow text + image content items for dynamic tool outputs (#10567) 2026-02-04 16:12:47 -08:00
EventMsg.json Support multimodal custom tool outputs (#12948) 2026-02-26 18:17:46 -08:00
ExecCommandApprovalParams.json chore: remove codex-core public protocol/shell re-exports (#12432) 2026-02-20 23:45:35 -08:00
ExecCommandApprovalResponse.json feat: include available decisions in command approval requests (#12758) 2026-02-26 01:10:46 +00:00
FileChangeRequestApprovalParams.json feat: vendor app-server protocol schema fixtures (#10371) 2026-02-01 23:38:43 -08:00
FileChangeRequestApprovalResponse.json feat: vendor app-server protocol schema fixtures (#10371) 2026-02-01 23:38:43 -08:00
FuzzyFileSearchParams.json feat: vendor app-server protocol schema fixtures (#10371) 2026-02-01 23:38:43 -08:00
FuzzyFileSearchResponse.json feat: vendor app-server protocol schema fixtures (#10371) 2026-02-01 23:38:43 -08:00
FuzzyFileSearchSessionCompletedNotification.json [app-server] add fuzzyFileSearch/sessionCompleted (#11773) 2026-02-13 15:08:14 -08:00
FuzzyFileSearchSessionUpdatedNotification.json app-server: add fuzzy search sessions for streaming file search (#10268) 2026-02-12 10:49:44 -08:00
JSONRPCError.json feat: vendor app-server protocol schema fixtures (#10371) 2026-02-01 23:38:43 -08:00
JSONRPCErrorError.json feat: vendor app-server protocol schema fixtures (#10371) 2026-02-01 23:38:43 -08:00
JSONRPCMessage.json feat: vendor app-server protocol schema fixtures (#10371) 2026-02-01 23:38:43 -08:00
JSONRPCNotification.json feat: vendor app-server protocol schema fixtures (#10371) 2026-02-01 23:38:43 -08:00
JSONRPCRequest.json feat: vendor app-server protocol schema fixtures (#10371) 2026-02-01 23:38:43 -08:00
JSONRPCResponse.json feat: vendor app-server protocol schema fixtures (#10371) 2026-02-01 23:38:43 -08:00
RequestId.json feat: vendor app-server protocol schema fixtures (#10371) 2026-02-01 23:38:43 -08:00
ServerNotification.json feat(app-server): thread/unsubscribe API (#10954) 2026-02-25 13:14:30 -08:00
ServerRequest.json feat: include available decisions in command approval requests (#12758) 2026-02-26 01:10:46 +00:00
ToolRequestUserInputParams.json feat: vendor app-server protocol schema fixtures (#10371) 2026-02-01 23:38:43 -08:00
ToolRequestUserInputResponse.json feat: vendor app-server protocol schema fixtures (#10371) 2026-02-01 23:38:43 -08:00