Support tool calling while streaming Gemini responses - #5
Merged
Conversation
ActuallyTaylor
force-pushed
the
feature/gemini-streaming-tool-calling
branch
from
August 3, 2026 19:36
899d9fa to
fa16ae2
Compare
ActuallyTaylor
force-pushed
the
feature/gemini-streaming-tool-calling
branch
from
August 3, 2026 19:43
fa16ae2 to
3777fa8
Compare
Gemini's streamResponse only handled .text parts, so .functionCall parts arriving over SSE were silently dropped and client tools never fired when streaming — the same prompt worked through respond but not through streamResponse. Wrap the request in a turn loop that accumulates both assistant text and any function-call parts for a turn. When a turn ends with calls pending, execute them through the existing resolveFunctionCalls, append the model turn (text plus functionCall parts) and a single user turn carrying the functionResponse parts, then issue another request. The loop exits when the model finishes a turn without asking for a tool. Transcript updates are live rather than deferred to the end: growStreamingTranscript as text arrives, and the .toolCalls entry is appended before the .toolOutput entries of the same turn so a Transcript-driven UI renders them in order. A .stop resolution appends the .toolCalls entry, finishes the continuation, and returns without executing. Snapshot yielding (String versus structured partial-JSON handling) is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Mirrors the Anthropic suite's streamWithTools: asserts that both a tool call and a tool output appear in the session transcript while the stream is still being consumed, not just after it completes, and that getWeather is present in the final transcript. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Newer Gemini models attach an opaque `thoughtSignature` to each `functionCall` part and require it back verbatim on the follow-up turn. The part decoder dropped it, so the second request of any tool-calling turn failed with `INVALID_ARGUMENT`: "Function call is missing a thought_signature in functionCall parts." The signature sits beside `functionCall` rather than inside it, so `GeminiPart` now reads it during decode, carries it on `GeminiFunctionCall`, and writes it back out during encode. Pattern matches on `.functionCall(let call)` are unaffected. Also moves the test suite off `gemini-2.5-flash`, which now returns 404 "no longer available to new users" for keys that were not already using it. Found by running the tool tests against the live API.
ActuallyTaylor
force-pushed
the
feature/gemini-streaming-tool-calling
branch
from
August 3, 2026 20:03
3777fa8 to
3107112
Compare
ActuallyTaylor
commented
Aug 3, 2026
ActuallyTaylor
left a comment
ActuallyTaylor
left a comment
Collaborator
Author
There was a problem hiding this comment.
Looks good, it is different than Anthropic in how it breaks the loop but it seems to match the Gemini api
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
streamResponsehandled only.textparts, sofunctionCallparts were silently dropped and tools never fired when streaming.respondalready had a complete loop.Streaming now runs a turn loop over a locally maintained
contentsarray: it collects function calls alongside text, executes them through the sharedresolveToolCalls, appends the model turn and a user turn carrying thefunctionResponseparts, and re-requests until a turn ends without calls. Tool activity reaches the transcript while the stream is running, with the tool calls entry ahead of its outputs, and a.stopdecision finishes the stream without executing anything.Two bugs fixed along the way, both found by running against the live API:
thoughtSignatureto eachfunctionCalland require it echoed back verbatim; without it the second turn of every tool call failed withINVALID_ARGUMENT. It sits besidefunctionCallrather than inside it, soGeminiPartnow reads it on decode, carries it onGeminiFunctionCall, and writes it back on encode.breakinside aswitchin the turn loop would have exited the switch rather than the loop, re-sending identicalcontentsforever. The loop is now labeled.The suite also moves off
gemini-2.5-flash, which returns 404 "no longer available to new users" for keys not already using it.Unlike Anthropic, continuation can't be gated on a stop reason — Gemini reports
finishReason: "STOP"even on turns that request a call — so it gates on whether anyfunctionCallpart was seen.Verified live:
streamWithToolspasses.Based on #4; merge that first.
🤖 Generated with Claude Code