Support tool calling while streaming Open Responses - #8
Conversation
3f72c2c to
6925e25
Compare
0caa6e0 to
6adbfe4
Compare
6925e25 to
28e5c10
Compare
99059d2 to
0e2a140
Compare
7f169ce to
fe38569
Compare
0e2a140 to
cf48b30
Compare
fe38569 to
c5ed57b
Compare
cf48b30 to
2710424
Compare
c5ed57b to
0e7e7de
Compare
b7678a9 to
0e7e60f
Compare
0e7e60f to
ea6e582
Compare
The streaming path sent the session's tools but its event loop only handled text deltas and completion, so function-call output items were dropped and tools never fired when streaming. `OpenResponsesStreamEvent` now decodes `response.output_item.added`, `response.function_call_arguments.delta`, and `response.output_item.done`. Streamed items accumulate per output index, argument fragments append to the item the server already opened, and a function call is only surfaced once its arguments parse as JSON. Assembled calls run through the shared `resolveToolCalls`; the turn's output items are echoed back into the input list the same way the non-streaming path echoes `resp.output`, tool outputs follow as `.tool` messages, and another request starts until a turn ends without calls. Tool activity reaches the transcript while the stream is still running, with the tool calls entry ahead of its outputs, and a `.stop` decision finishes the stream without executing anything.
The suite already takes its endpoint from the environment but hardcoded `gpt-4o-mini`, so it could only ever run against OpenAI. Any server implementing the Responses API serves its own model names. `OPEN_RESPONSES_MODEL` now overrides the model, defaulting to `gpt-4o-mini` so existing setups are unaffected. Ollama, for example, runs the suite with `OPEN_RESPONSES_BASE_URL=http://localhost:11434/v1` and one of its local models.
ea6e582 to
7b1327c
Compare
|
Removed from the stack (now stack huggingface#15) and left open as a standalone PR against Status: the streaming implementation here is not verified end to end. Testing it against Ollama's Responses API surfaced a blocking bug that is not in this PR's streaming code: Tool The model is reliable — 5/5 tool calls once This is pre-existing and shared: The fix belongs in Non-tool paths of this PR do pass against Ollama ( 🤖 Generated with Claude Code |
ActuallyTaylor
left a comment
There was a problem hiding this comment.
Chilling. Same format as the rest of the implementations
The streaming path sent the session's tools but its event loop only handled text deltas and completion, so function-call output items were dropped and tools never fired when streaming.
OpenResponsesStreamEventnow decodesresponse.output_item.added,response.function_call_arguments.delta, andresponse.output_item.done. Streamed items accumulate per output index, argument fragments append to the item the server already opened, and a call is surfaced only once its arguments parse as JSON. Assembled calls run through the sharedresolveToolCalls; the turn's output items are echoed back into the input list the same way the non-streaming path echoesresp.output, tool outputs follow as.toolmessages, and another request starts until a turn ends without calls.Tool activity reaches the transcript while the stream is running, with the tool calls entry ahead of its outputs, and a
.stopdecision finishes the stream without executing anything..failedstill throwsstreamFailed.Note
Not verified against a live endpoint — this suite needs
OPEN_RESPONSES_API_KEYandOPEN_RESPONSES_BASE_URL, which weren't available. It is the one provider in this series still unproven end to end.Based on #4; merge that first.
🤖 Generated with Claude Code