Support tool calling while streaming OpenAI responses - #7
Merged
Conversation
ActuallyTaylor
force-pushed
the
feature/openai-streaming-tool-calling
branch
from
August 3, 2026 19:36
71c3da7 to
012a2c2
Compare
ActuallyTaylor
force-pushed
the
feature/openai-streaming-tool-calling
branch
2 times, most recently
from
August 3, 2026 19:47
a5003e9 to
7f169ce
Compare
ActuallyTaylor
changed the base branch from
feature/central-tool-resolution
to
feature/ollama-streaming-tool-calling
August 3, 2026 19:48
ActuallyTaylor
force-pushed
the
feature/openai-streaming-tool-calling
branch
2 times, most recently
from
August 3, 2026 20:07
fe38569 to
c5ed57b
Compare
Neither streaming variant handled tool calls. The Responses path had literal placeholders — `case .toolCallCreated` and `case .toolCallDelta` with comments saying tool call events are ignored — and the Chat Completions path never decoded `tool_calls` deltas at all, so tools silently never fired when streaming. Both variants now accumulate the fragments OpenAI streams: an opening event carries the call id and function name, later events append chunks of the arguments JSON. Fragments are keyed by index, and by `item_id` for the Responses variant since its argument deltas are keyed by item rather than position. A call is only surfaced once its fragments parse as JSON. Assembled calls run through the shared `resolveToolCalls`, the assistant turn and its tool results are appended, and another request starts until a turn ends without tool calls. `response.completed` now breaks the event loop rather than finishing the continuation, which would otherwise end the stream on the first tool-calling turn instead of starting the next request. Failures are no longer silent: the Responses endpoint answers with HTTP 200 and reports errors as `error` or `response.failed` events, both of which previously fell through to `.ignored`, so a failed request produced an empty response with no error. They now throw `OpenAILanguageModelError.streamFailed` carrying the API's message. Verified against the live API: tool tests pass for both variants.
ActuallyTaylor
force-pushed
the
feature/openai-streaming-tool-calling
branch
from
August 3, 2026 20:09
c5ed57b to
0e7e7de
Compare
`.init` leaves the reader to infer the type from context, which is hardest to do exactly where this file used it most: inside nested literals and behind a `Content.Type` metatype. Every site now names its type — `OpenAIMessage(...)` in the transcript conversion, `Transcript.TextSegment(...)` in the refusal path, `LanguageModelSession.ResponseStream<Content>.Snapshot(...)` for streamed snapshots, and `Content(...)` in place of `type.init(...)`. The doc comment example names `CustomGenerationOptions` too, so it can be copied without the surrounding assignment. No behavior change; the live OpenAI tool tests pass for both API variants.
ActuallyTaylor
commented
Aug 3, 2026
ActuallyTaylor
left a comment
ActuallyTaylor
left a comment
Collaborator
Author
There was a problem hiding this comment.
Looks good
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Neither streaming variant handled tool calls. The Responses path had literal placeholders —
case .toolCallCreatedandcase .toolCallDeltawith comments saying tool call events are ignored — and Chat Completions never decodedtool_callsdeltas at all.Both variants now accumulate the fragments OpenAI streams: an opening event carries the call id and function name, later events append chunks of the arguments JSON. Fragments are keyed by index, and by
item_idfor the Responses variant since its argument deltas are keyed by item rather than position. A call is surfaced only once its fragments parse as JSON. Assembled calls run through the sharedresolveToolCalls, then the assistant turn and its tool results are appended and another request starts, until a turn ends without calls.response.completednow breaks the event loop rather than finishing the continuation, which would otherwise end the stream on the first tool-calling turn instead of starting the next request.Failures are no longer silent. The Responses endpoint answers with HTTP 200 and reports errors as
errororresponse.failedevents. Both previously fell through to.ignored, so a failed request produced an empty response indistinguishable from a model that said nothing. They now throwOpenAILanguageModelError.streamFailedcarrying the API's own message — found when an exhausted-quota account reported empty output instead of "You have no credits remaining."Verified live: tool tests pass for both variants.
Based on #4; merge that first.
🤖 Generated with Claude Code