Non-buffered Chat Completions tool-call streaming finalizes incomplete tool calls with empty name/call_id

[baiyuxi930826]

Please read this first

• Have you read the docs? Agents SDK docs
• Have you searched for related issues? Searched for Chat Completions streaming / buffer_streamed_tool_calls / incomplete tool-call handling; did not find an existing report of this default-vs-buffered asymmetry.

Describe the bug

For OpenAI Chat Completions streaming tool calls, the default path (buffer_streamed_tool_calls=False) and the buffered path (buffer_streamed_tool_calls=True) disagree on incomplete tool-call streams:

Path	Incomplete tool call (missing id and/or function name)
Default / non-buffered	Still finalizes a ResponseFunctionToolCall with empty name and/or call_id via the end-of-stream fallback
Buffered	Raises ModelBehaviorError (without a tool call id / without a function name)

The same incomplete upstream stream can either hard-fail or silently produce an unusable (or partially usable) tool call, depending only on a flag that defaults to False. LiteLLM/AnyLLM integrations always use the non-buffered path today.

This appears to come from ChatCmplStreamHandler.handle_stream:

1. On first tool-call delta for an index, it seeds ResponseFunctionToolCall(name="", call_id="", ...).
2. Live streaming only starts once both name and call_id are present.
3. At stream end, if streaming never started, the "fallback to old behavior" branch still emits response.output_item.added / arguments delta / response.output_item.done with those empty fields.

The buffered path validates completeness in _buffered_tool_call_delta and raises instead (added in PR #3506).

Debug information

• Agents SDK version: 0.18.3 (checkout 965335aba6f6c71500e0b8cdb4e9e495f5801d4d)
• Python version: Python 3.12.10
• OS: Windows 11
• API surface: Chat Completions streaming (OpenAIChatCompletionsModel / OpenAIProvider(use_responses=False))
• Flag: buffer_streamed_tool_calls default False vs True
• Reproduction count: 2/2 stable (unit-level synthetic stream; no live API key)

Repro steps

Minimal unit-style repro against the stream handler (no live API key required):

import asyncio
from typing import AsyncIterator

from openai.types.chat.chat_completion_chunk import (
    ChatCompletionChunk,
    Choice,
    ChoiceDelta,
    ChoiceDeltaToolCall,
    ChoiceDeltaToolCallFunction,
)
from openai.types.responses import Response

from agents.models.chatcmpl_stream_handler import ChatCmplStreamHandler
from agents.exceptions import ModelBehaviorError


def _chunk(tool_calls=None, finish_reason=None) -> ChatCompletionChunk:
    return ChatCompletionChunk(
        id="chunk-id",
        created=1,
        model="fake",
        object="chat.completion.chunk",
        choices=[
            Choice(
                index=0,
                delta=ChoiceDelta(tool_calls=tool_calls),
                finish_reason=finish_reason,
            )
        ],
    )


async def incomplete_stream() -> AsyncIterator[ChatCompletionChunk]:
    # Incomplete: index + partial args, but no tool call id and no function name.
    yield _chunk(
        tool_calls=[
            ChoiceDeltaToolCall(
                index=0,
                function=ChoiceDeltaToolCallFunction(arguments='{"a":'),
            )
        ]
    )
    yield _chunk(finish_reason="tool_calls")


async def main() -> None:
    response = Response(
        id="resp",
        created_at=1,
        model="fake",
        object="response",
        output=[],
        parallel_tool_calls=False,
        tool_choice="auto",
        tools=[],
    )

    # Default / non-buffered
    events = [
        event
        async for event in ChatCmplStreamHandler.handle_stream(
            response, incomplete_stream()
        )
    ]
    done_items = [
        e.item
        for e in events
        if getattr(e, "type", None) == "response.output_item.done"
    ]
    print("non-buffered done items:", done_items)
    # Observed: function_call with name="" and call_id=""

    # Buffered
    try:
        async for _ in ChatCmplStreamHandler.handle_stream(
            response,
            ChatCmplStreamHandler.buffer_tool_call_stream(incomplete_stream()),
        ):
            pass
        print("buffered: no error")
    except ModelBehaviorError as exc:
        print("buffered raised:", type(exc).__name__, str(exc))
        # Observed: ModelBehaviorError about missing tool call id / function name


asyncio.run(main())

Related existing coverage asserts the buffered hard-fail only:

• tests/models/test_openai_chatcompletions_stream.py::test_buffered_tool_call_delta_requires_id_and_name
• tests/models/test_openai_chatcompletions_stream.py::test_stream_response_buffered_tool_calls_raise_for_missing_tool_call_delta

I did not find a matching assertion that the default path rejects the same incomplete stream.

Expected behavior

Buffered and non-buffered Chat Completions streaming paths should treat incomplete tool-call streams consistently.

If a tool call never receives a name and/or call_id, the default non-buffered path should fail closed the same way as the buffered path (ModelBehaviorError), rather than emitting a finalized function call with empty identifiers.

(If empty finalization is intentionally kept for backward compatibility, that should be documented, and ideally warned/gated rather than silent.)

Actual behavior

• Non-buffered (default): end-of-stream fallback finalizes incomplete tool calls with empty name / call_id.
• Buffered: raises ModelBehaviorError when id or name is missing.

Additional observation from runner path analysis: empty name tends to fail later as "tool not found"; a present name with empty call_id can still reach tool execution with an empty call_id, which may break correlation/history pairing.

Possible cause

This may be legacy fallback behavior in ChatCmplStreamHandler.handle_stream (comment about "function name never arrived"), while buffer_tool_call_stream later added strict completeness checks in PR #3506. The safer checks are behind a non-default flag, so default and LiteLLM remain on the permissive finalizer.

Relevant spots:

• src/agents/models/chatcmpl_stream_handler.py — empty seed; stream gate; end-of-stream fallback
• src/agents/models/chatcmpl_stream_handler.py — _buffered_tool_call_delta validation
• src/agents/models/openai_chatcompletions.py — buffer_streamed_tool_calls: bool = False

Suggested direction

1. Align non-buffered finalization with buffered validation (raise ModelBehaviorError when name or call_id is still empty at stream end), or
2. If empty finalization must remain for compatibility, document it and consider a warning / opt-in strict mode; prefer fail-closed by default for incomplete identifiers.
3. Add a regression test that feeds incomplete tool-call deltas through both paths and asserts matching failure (or explicit drop) behavior.
4. Optional defense-in-depth: reject empty call_id before tool execution in the runner.

I'd be happy to prepare a PR with a regression test if this behavior is confirmed.

Additional context

• Related prior art (not a duplicate bug report): PR feat: add buffered Chat Completions tool-call streaming #3506
• Nearby but different: chat completion model failed to produce multiple tool call items #989, Chat Completions streaming fallback tool calls can reuse output_index #3104 / fix: #3104 stabilize chat completions tool call output indexes #3161 (stream/tool indexing)

[seratch]

Could you provide a redacted raw stream from a real provider that ends without the ID or function name, together with the provider, model, adapter (OpenAIChatCompletionsModel, LiteLLM, or AnyLLM), versions, and observed downstream failure? For OpenAIProvider, please also confirm whether buffer_streamed_tool_calls=True avoids the problem. That evidence will determine whether a fix belongs in a specific adapter or the shared finalization path.

[baiyuxi930826]

Thanks for the review — here is the additional evidence you asked for.

What we can share today

We reproduced this deterministically without a live OpenAI key by feeding incomplete Chat Completions chunks into the same conversion layer production streaming uses: ChatCmplStreamHandler.handle_stream (also used by LiteLLM/AnyLLM, which never enable buffering).

Environment

• Agents SDK: 0.18.3 @ 965335aba6f6c71500e0b8cdb4e9e495f5801d4d
• openai Python package: 2.46.0
• OS: Windows 11 / Python 3.12.10
• Adapter path: Chat Completions stream handler (shared by OpenAIChatCompletionsModel, LiteLLM, AnyLLM)

Incomplete stream shape (synthetic, redacted of any secrets)

Equivalent chunk sequence (arguments-only tool_call delta, then finish_reason=tool_calls, no id / function name):

data: {"id":"chunk-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"type":"function","function":{"arguments":"{\"a\":"}}]},"finish_reason":null}]}

data: {"id":"chunk-2","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}]}

data: [DONE]

(Real providers that interleave id/name on later packets, or cut the TCP/SSE stream after argument deltas, produce the same incomplete state at end-of-stream.)

Results (2/2 stable)

Mode	Result
Default non-buffered (buffer_streamed_tool_calls=False)	Finalizes ResponseFunctionToolCall(name="", call_id="") via end-of-stream fallback; no ModelBehaviorError
Buffered (buffer_tool_call_stream / buffer_streamed_tool_calls=True)	ModelBehaviorError: Buffered Chat Completions tool call stream ended without a tool call id.

So for OpenAIProvider with buffer_streamed_tool_calls=True: yes — that path avoids silent empty finalization and fails closed, matching what you asked to confirm.

Downstream impact observed in code path analysis

• Empty name → later dispatch typically surfaces as tool-not-found (Tool not found…).
• Present name + empty call_id can still reach tool execution with call_id="" (correlation/history risk).
• LiteLLM/AnyLLM always use the non-buffered handler today, so they stay on the permissive finalization path.

Why this still seems like the shared finalization path

Buffered validation lives in _buffered_tool_call_delta (PR #3506), while the default finalizer still emits incomplete items. A real flaky/truncated provider stream would hit the same shared fallback regardless of adapter, unless that adapter opts into buffering.

I do not currently have a production customer capture of a third-party gateway dropping id/name mid-stream. If you prefer to gate a code change on a live provider capture, I can try to obtain one next; otherwise I’m happy to open a PR that:

1. fail-closes incomplete name/call_id on the non-buffered finalizer (parity with buffered), and
2. adds a unit regression on synthetic incomplete chunks (no network).

Happy to adjust the PR scope based on your preference (shared finalizer vs adapter-only).

[seratch]

Thanks for the additional details. This confirms the synthetic handler behavior and that buffer_streamed_tool_calls=True fails closed, but it does not yet establish the real provider path we asked about. The stream is still manually constructed, and the statements about providers reordering fields or a truncated connection ending in the same state are not backed by a named provider capture or transport-level reproduction.

Please hold off on opening a PR until you can provide either a redacted raw stream from a named provider or a reproduction showing that a real transport stream ends cleanly without the ID/name, together with the observed runner outcome. That evidence is necessary because the current fallback is released compatibility behavior. Once we have it, we can decide whether the fix belongs in a specific adapter or the shared finalizer.