Skip to content

Support tool calling while streaming Open Responses - #8

Merged
ActuallyTaylor merged 2 commits into
mainfrom
feature/openresponses-streaming-tool-calling
Aug 3, 2026
Merged

Support tool calling while streaming Open Responses#8
ActuallyTaylor merged 2 commits into
mainfrom
feature/openresponses-streaming-tool-calling

Conversation

@ActuallyTaylor

Copy link
Copy Markdown
Collaborator

The streaming path sent the session's tools but its event loop only handled text deltas and completion, so function-call output items were dropped and tools never fired when streaming.

OpenResponsesStreamEvent now decodes response.output_item.added, response.function_call_arguments.delta, and response.output_item.done. Streamed items accumulate per output index, argument fragments append to the item the server already opened, and a call is surfaced only once its arguments parse as JSON. Assembled calls run through the shared resolveToolCalls; the turn's output items are echoed back into the input list the same way the non-streaming path echoes resp.output, tool outputs follow as .tool messages, and another request starts until a turn ends without calls.

Tool activity reaches the transcript while the stream is running, with the tool calls entry ahead of its outputs, and a .stop decision finishes the stream without executing anything. .failed still throws streamFailed.

Note

Not verified against a live endpoint — this suite needs OPEN_RESPONSES_API_KEY and OPEN_RESPONSES_BASE_URL, which weren't available. It is the one provider in this series still unproven end to end.

Based on #4; merge that first.

🤖 Generated with Claude Code

@ActuallyTaylor
ActuallyTaylor force-pushed the feature/central-tool-resolution branch from 3f72c2c to 6925e25 Compare August 3, 2026 19:35
@ActuallyTaylor
ActuallyTaylor force-pushed the feature/openresponses-streaming-tool-calling branch from 0caa6e0 to 6adbfe4 Compare August 3, 2026 19:36
@ActuallyTaylor
ActuallyTaylor force-pushed the feature/central-tool-resolution branch from 6925e25 to 28e5c10 Compare August 3, 2026 19:39
@ActuallyTaylor
ActuallyTaylor force-pushed the feature/openresponses-streaming-tool-calling branch 2 times, most recently from 99059d2 to 0e2a140 Compare August 3, 2026 19:47
@ActuallyTaylor
ActuallyTaylor changed the base branch from feature/central-tool-resolution to feature/openai-streaming-tool-calling August 3, 2026 19:48
@ActuallyTaylor
ActuallyTaylor force-pushed the feature/openai-streaming-tool-calling branch from 7f169ce to fe38569 Compare August 3, 2026 20:03
@ActuallyTaylor
ActuallyTaylor force-pushed the feature/openresponses-streaming-tool-calling branch from 0e2a140 to cf48b30 Compare August 3, 2026 20:03
@ActuallyTaylor
ActuallyTaylor force-pushed the feature/openai-streaming-tool-calling branch from fe38569 to c5ed57b Compare August 3, 2026 20:07
@ActuallyTaylor
ActuallyTaylor force-pushed the feature/openresponses-streaming-tool-calling branch from cf48b30 to 2710424 Compare August 3, 2026 20:07
@ActuallyTaylor
ActuallyTaylor force-pushed the feature/openai-streaming-tool-calling branch from c5ed57b to 0e7e7de Compare August 3, 2026 20:09
@ActuallyTaylor
ActuallyTaylor force-pushed the feature/openresponses-streaming-tool-calling branch 2 times, most recently from b7678a9 to 0e7e60f Compare August 3, 2026 20:17
Base automatically changed from feature/openai-streaming-tool-calling to main August 3, 2026 20:18
@ActuallyTaylor
ActuallyTaylor force-pushed the feature/openresponses-streaming-tool-calling branch from 0e7e60f to ea6e582 Compare August 3, 2026 20:18
claude added 2 commits August 3, 2026 16:39
The streaming path sent the session's tools but its event loop only handled text deltas and completion, so function-call output items were dropped and tools never fired when streaming.

`OpenResponsesStreamEvent` now decodes `response.output_item.added`, `response.function_call_arguments.delta`, and `response.output_item.done`. Streamed items accumulate per output index, argument fragments append to the item the server already opened, and a function call is only surfaced once its arguments parse as JSON. Assembled calls run through the shared `resolveToolCalls`; the turn's output items are echoed back into the input list the same way the non-streaming path echoes `resp.output`, tool outputs follow as `.tool` messages, and another request starts until a turn ends without calls.

Tool activity reaches the transcript while the stream is still running, with the tool calls entry ahead of its outputs, and a `.stop` decision finishes the stream without executing anything.
The suite already takes its endpoint from the environment but hardcoded `gpt-4o-mini`, so it could only ever run against OpenAI. Any server implementing the Responses API serves its own model names.

`OPEN_RESPONSES_MODEL` now overrides the model, defaulting to `gpt-4o-mini` so existing setups are unaffected. Ollama, for example, runs the suite with `OPEN_RESPONSES_BASE_URL=http://localhost:11434/v1` and one of its local models.
@ActuallyTaylor
ActuallyTaylor force-pushed the feature/openresponses-streaming-tool-calling branch from ea6e582 to 7b1327c Compare August 3, 2026 20:40
@ActuallyTaylor

Copy link
Copy Markdown
Collaborator Author

Removed from the stack (now stack huggingface#15) and left open as a standalone PR against main, to be picked up after the schema issue below is resolved.

Status: the streaming implementation here is not verified end to end. Testing it against Ollama's Responses API surfaced a blocking bug that is not in this PR's streaming code:

Tool parameters are sent with a $defs block. GenerationSchema.withResolvedRoot() inlines the root schema but leaves the now-unreferenced definitions behind. Replaying the exact request two ways against Ollama:

with $defs → ['reasoning', 'message']        ← no tool call
without    → ['reasoning', 'function_call']  ← tool called

The model is reliable — 5/5 tool calls once $defs is dropped — so the server silently declines to call a tool whose parameters carry it.

This is pre-existing and shared: withResolvedRoot() lives in GenerationSchema.swift, and OpenAI, Ollama, Gemini, MLX, Anthropic and System all use the same pattern. It goes unnoticed against OpenAI's own API, which tolerates $defs.

The fix belongs in withResolvedRoot() — after inlining the root, drop defs no longer reachable from it. Deleting $defs wholesale would break nested @Generable types that genuinely $ref into it.

Non-tool paths of this PR do pass against Ollama (basicResponse, streaming, streamingString, withInstructions).

🤖 Generated with Claude Code

@ActuallyTaylor ActuallyTaylor left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chilling. Same format as the rest of the implementations

@ActuallyTaylor
ActuallyTaylor merged commit 6fd47bb into main Aug 3, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants