Repository navigation
fix(openai): keep streaming when chunks include usage - #13418
Open
cestercian wants to merge 1 commit into
Open
cestercian wants to merge 1 commit into
cestercian wants to merge 1 commit into
Conversation
Gemini's OpenAI-compat endpoint sends usage on every SSE chunk, next to content and tool call deltas. The adapter treated any usage payload as the final chunk and dropped the rest, so chat text and tool calls never showed up. Keep yielding those deltas, pass through extra fields like extra_content, and let the latest usage win.
Contributor
|
All contributors have signed the CLA ✍️ ✅ |
Author
|
I have read the CLA Document and I hereby sign the CLA |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Gemini's OpenAI compatible endpoint sends
usageon every SSE chunk, right next to the content and tool call deltas. The stream handling treated any chunk with usage as the final one, so text got dropped and tool calls never showed up.Now the deltas keep streaming when usage is present, the latest usage wins, and unknown extras like
extra_contentare passed through instead of breaking things. Normal OpenAI behavior (usage only on the last empty-choices chunk) is unchanged.Fixes #13417
Checklist
Tests
packages/openai-adapters/src/test/openai-gemini-usage.vitest.ts: Gemini style chunks with usage on every chunk plus content and tool call deltascore/llm/openaiTypeConverters.gemini.vitest.ts: chunk conversion keeps content, tool calls and usage together