chat: keep vendor fields on tool calls in both directions - #92
Merged
Merged
Conversation
Gemini 3 on its OpenAI-compatible endpoint returns a thought signature at tool_calls[i].extra_content.google.thought_signature and rejects the next turn with 400 INVALID_ARGUMENT when the echoed call lacks it. The typed ToolCall (id/type/function) dropped the field at three sites: the request parse, the buffered response render, and the stream accumulator that the DLP-hit replay rebuilds from. A tool loop through the gateway failed on turn 2 with 424. ChatMessage.tool_calls is now raw JSON like ChunkDelta.tool_calls and ChatMsg.tool_calls: the request moves into ChatMsg without the per-call to_value copy, and the buffered render forwards what the engine returned. The accumulator keeps the first value of every fragment field except index and function. The typed render's 500 arm goes with it; nothing converts, so nothing can drop.
Raw chat tool calls can carry non-object values after vendor-field passthrough. Reject those shapes as a client error before Responses conversion and move valid fields without reserializing them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On Google's OpenAI-compatible endpoint (
/v1beta/openai), Gemini 3 returns a per-call thought signature attool_calls[i].extra_content.google.thought_signature. The next turn must echo it back, or Google answers400 INVALID_ARGUMENT: Function call is missing a thought_signature. The gateway dropped the field in three places, so every Gemini 3 tool loop through it failed on turn 2 with424(original_status_code: 400).ChatMessage.tool_calls: Vec<ToolCall>, thento_valuefrom_value::<Vec<ToolCall>>stream: falsemerge_tool_call_fragmentscopies only id/type/name/argumentsThe live stream path and
wire_messagesalready forwarded the field.Fix
ChatMessage.tool_callsbecomesOption<Vec<Value>>, the same shapeChunkDelta.tool_callsandChatMsg.tool_callsalready use.ToolCallandFunctionCallhad no other users, so they are removed.m.tool_calls.map(Value::Array)moves the calls. This removes the old per-requestto_value, which re-serialized every historical call, arguments included.openai_tool_calls(..)unchanged. The chat: render anthropic tool_use blocks as OpenAI tool_calls #28500arm existed because the typed conversion could fail, and before chat: render anthropic tool_use blocks as OpenAI tool_calls #28 that failure fell back to[]. With no conversion there is nothing to drop, and the stream surface already forwarded the same payload.indexandfunction. Insidefunction, the name is still first-wins and arguments still append.Rejected alternatives:
extra_content: Option<Value>onToolCallkeeps the typed round trip. Every request would then copy each historical signature once more, becauseto_valueserializes by reference, and the next vendor field would be dropped again.#[serde(flatten)]buffers every call through serde'sContenton every request.Ownership
ChatMsg, thenwire_messages, then the upstream body.GatewayResponse.tool_callsinto the response with no clone.id/typeclone.Behavior changes
type, objectarguments) is no longer rejected with a gateway400. It reaches the vendor, which returns a 4xx. That does not trigger fallback, becauseis_upstream_faultfalls back only on 5xx/429.anthropic-messagesmodel now keep theindexthattool_use_to_tool_callssets; the typed parse used to strip it. I probed OpenAI, DeepSeek, DashScope, SiliconFlow and Moonshot directly: all acceptindexandextra_contenton an echoed call.security_checkwalks every string leaf, opaque proofs included (SignedThinking::Visit). The signature is now scanned the same way an Anthropic thinking signature already is.@, secrets need-/_orAKIA[0-9A-Z]{16}, and the CN-mobile rule needs1[3-9]followed by 9 more digits.Hot path
The A/B ran on an Apple M2 Max, with both versions of
openai.rsin one process via#[path]. Each case ran 41 rounds with alternating arm order; the whole run was then repeated with the order swapped. The table shows the mean of the two runs' medians. The two runs agree within 2%.Evidence
tool_calls_keep_vendor_fields_both_ways(protocol),tool_call_fragments_keep_vendor_fields_first_wins(engine), andchat_tool_calls_keep_vendor_fields_both_ways(e2e: buffered, live stream, and DLP-hit replay). Each one fails on the old code:srcat main, the e2e fails on the upstream-body assertion (left: Null);dlp=trueonly;cargo fmt --check,cargo clippy --workspace --all-targets -- -D warningsandcargo test --workspaceare clean: 31 suites, 622 passed, 0 failed. The env-gated PG/Redis suites were not run.gemini-3.6-flashtool loop (reasoning_effort: high) through a local gateway. The client echoes the turn-1 assistant message verbatim. The branch passed 3 of 3 runs in each mode, plus the side-by-side run in the table.0c087b99extra_contentstop424(upstream 400)stopextra_contentstop424(upstream 400)stopNot in this PR
geminiprotocol engine (VertexEngine) has no tool calling:toolsnever reaches the body andfunctionCallparts are not parsed.openai-chatagainst/v1beta/openaiis the only Gemini tool route.index(pre-existing; visible in this PR's e2e output). Strict stream parsers such as the AI SDK's zod schema reject those deltas./v1/messagesserved by anopenai-chatmodel cannot carry the signature, because Anthropictool_useblocks have no slot for it.