Skip to content

feat(openai): surface Gemini thinking + request thought summaries - #301

Open
philipph-askui wants to merge 1 commit into
mainfrom
feat/openai-reasoning-narration
Open

feat(openai): surface Gemini thinking + request thought summaries#301
philipph-askui wants to merge 1 commit into
mainfrom
feat/openai-reasoning-narration

Conversation

@philipph-askui

@philipph-askui philipph-askui commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

The problem (conceptually)

A reasoning model keeps its "thinking" separate from its spoken answer. Claude returns that reasoning as a distinct, typed thinking block; the OpenAI-compatible path (how Gemini is served via the AskUI API) is messier and, as a result, dropped the reasoning entirely:

  1. We never asked for the thoughts. Gemini (via Vertex's OpenAI-compatible endpoint) only returns a thought summary if you explicitly opt in. We weren't opting in.
  2. Even when thoughts came back, nothing extracted them. The summary arrives either in a side field (reasoning_content / reasoning) or — for Gemini — inline in the answer text, wrapped in <think>…</think> tags. The parser ignored the side field and passed the inline tags straight through as part of the answer.

So a Python-SDK user running Gemini through the AskUI API saw no reasoning at all, or raw <think> tags leaking into the answer. (There's also a sharp edge: Vertex 400s if a request carries both reasoning_effort and a custom thinking_config"found both" — so the opt-in must use thinking_config alone.)

This path previously only carried the Gemini thought_signature (multi-turn tool calling, #290); reasoning was never surfaced.

The fix (conceptually)

Make the OpenAI-compatible path speak the same internal thinking vocabulary as the Anthropic path:

  • Ask for the thoughts on the AskUI Gemini path (extra_body.google.thinking_config.include_thoughts, deliberately not reasoning_effort).
  • Recognize reasoning in both forms in the response parser — the side field and inline <think>…</think> — and lift either into the same BetaThinkingBlock the Anthropic path already produces, ahead of the spoken text.

Changes

  • models/openai/messages_api.py — parse reasoning_content / reasoning; split inline <think>…</think> out of content; emit a BetaThinkingBlock before the text block.
  • model_providers/askui_vlm_provider.py — on the Google backend, request thoughts via extra_body.google.thinking_config.include_thoughts (caller provider_options win).
  • Unit tests for both parse forms + the request wiring (55 passed on the affected files); ruff + mypy clean.

Related PRs

This is a parity port of the C# SDK change:

  • C# SDK (original fix): askui/csharp-sdk#13
  • Desktop app (renders the result): askui/integrated-task-platform#177 — where the fix was validated against a live Gemini run.

The desktop app runs on the C# SDK, so it does not depend on this PR; this is a standalone-product parity fix for Python-SDK users.

🤖 Generated with Claude Code

The OpenAI-compatible path only carried the Gemini thought_signature; it
never surfaced the model's reasoning. Two additions, mirroring the C# SDK
change (askui/csharp-sdk#13):

- Parse reasoning into a thinking block: read `reasoning_content` / `reasoning`
  from the response message's extra fields, and split inline `<think>…</think>`
  out of `content` (how Gemini via Vertex's OpenAI endpoint returns its thought
  summary). Either becomes a BetaThinkingBlock ahead of the spoken text.
- Request the summary on the AskUI Gemini path via
  `extra_body.google.thinking_config.include_thoughts`. Vertex rejects
  `reasoning_effort` and a custom `thinking_config` together ("found both",
  400), so only the latter is sent; caller-supplied options win.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@philipph-askui
philipph-askui marked this pull request as ready for review July 28, 2026 20:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant