Skip to content

Federated tool errors from the anyio/httpx transport family surface as an empty "Tool invocation failed: " (str(root_cause) is ''); ExceptionGroup-wrapped timeouts bypass the typed timeout lanesΒ #6592

Open

Description

@godwinbw-egm

🐞 Bug Summary

Federated tools/call failures in the anyio/httpx transport family reach the caller as the bare string Tool invocation failed: β€” an empty detail β€” because invoke_tool's outer handler builds the caller-facing message as error_message = str(root_cause) and str() of the whole transport family is ''. A caller cannot tell a read timeout from a closed stream from a broken resource. The exception class is logged server-side but never reaches the message.

A second, related gap: a read timeout raised inside the MCP SDK's TaskGroup arrives as an ExceptionGroup, so it misses the per-transport except (asyncio.TimeoutError, httpx.TimeoutException) lanes (which produce the nice typed ToolTimeoutError / Tool invocation timed out after {effective_timeout}s), hits the per-transport except BaseException (which logs type(root_cause).__name__ and re-raises the group unchanged), and lands in the outer handler above β€” where it stringifies to ''.


🧩 Affected Component

  • mcpgateway - API
  • mcpgateway - UI (admin panel)
  • mcpgateway.wrapper - stdio wrapper
  • Federation or Transports
  • CLI, Makefiles, or shell scripts
  • Container setup (Docker/Podman/Compose)
  • Other (explain below)

πŸ” Steps to Reproduce

A. The str() proof β€” inside the published image's own venv (v1.0.8):

docker run --rm --entrypoint /app/.venv/bin/python3 ghcr.io/ibm/mcp-context-forge:v1.0.8 -c "
import anyio, httpx
print([repr(str(e())) for e in (anyio.ClosedResourceError, anyio.EndOfStream, anyio.BrokenResourceError, anyio.WouldBlock)])
print(repr(str(httpx.ReadTimeout(''))))"
# ["''", "''", "''", "''"]
# ''

(anyio 4.14.2, httpx 0.28.1, Python 3.12.13 in that image.)

B. Live, through the gateway (v1.0.8): register a streamable-HTTP MCP backend whose tool runs longer than tool_timeout (default 60 s) and call it via POST /servers/{id}/mcp tools/call. Observed: HTTP 200 after ~65 s with

{"result":{"content":[{"type":"text","text":"Tool invocation failed: "}],"isError":true}}

The server log for the same call carries error_type = the real class; the caller gets nothing.

C. Where it happens (mcpgateway/services/tool_service.py):

  • v1.0.8 β€” outer except BaseException as e: of invoke_tool at :6716; error_message = str(root_cause) at :6723; emitted at :6740 (post-invoke ToolResult) and :6780 (raise ToolInvocationError(f"Tool invocation failed: {error_message}")).
  • main@1321848 (checked 2026-09-03) β€” same construction, now at :7029; still no fallback. The bare except (asyncio.TimeoutError, httpx.TimeoutException): lanes are at :5895, :6359, :6546, :6825 and still do not see an ExceptionGroup-wrapped timeout.

πŸ€” Expected Behavior

The caller-facing message should never be empty. Two small changes at the one construction site would do it (this is what we ship as a build-time patch on our pinned image):

error_message = str(root_cause) or f"{type(root_cause).__module__}.{type(root_cause).__qualname__}"
if isinstance(root_cause, (asyncio.TimeoutError, httpx.TimeoutException)):
    error_message = f"Tool invocation timed out after {effective_timeout}s ({type(root_cause).__module__}.{type(root_cause).__qualname__})"
  • str(root_cause) or <class> is byte-identical for every exception whose str() is already non-empty (e.g. httpx.HTTPStatusError for a 503), so nothing that works today changes.
  • The timeout branch reuses the wording the per-transport lanes already emit, so callers key on one vocabulary regardless of whether the timeout arrived group-wrapped. With this in place the live call in (B) returns Tool invocation failed: Tool invocation timed out after 60s (builtins.TimeoutError).

(The nested-group recursion while isinstance(root_cause, BaseExceptionGroup) and root_cause.exceptions: already unwraps to the leaf β€” thank you β€” so the leaf is the right thing to name.)


πŸ““ Logs / Error Output

Caller-facing (v1.0.8, unmodified image), read-timeout case:

Tool invocation failed: 

Same call after the change above:

Tool invocation failed: Tool invocation timed out after 60s (builtins.TimeoutError)

🧠 Environment Info

Key Value
Version or commit v1.0.8 (reproduced); construction unchanged at main@1321848
Runtime Python 3.12.13, Gunicorn (as shipped in ghcr.io/ibm/mcp-context-forge:v1.0.8)
Platform / OS Fly.io (linux/amd64) and local Docker Desktop (macOS)
Container Docker

🧩 Additional Context (optional)

The empty-str() family is a property of anyio/httpx, not of the gateway β€” but the gateway is the only place that can add the class before the message leaves. We currently carry this as a one-anchor exact-match patch applied at image build; happy to open a PR if the shape above is acceptable.

Activity

  1. godwinbw-egm commented on Sep 4, 2026

    @godwinbw-egm
    Author

    Follow-up β€” a second, related shape: a mid-call upstream disconnect is reported as Tool invocation timed out after {tool_timeout}s after the full budget.

    With mcp>=1.28.1,<2 (v1.0.8, v1.0.9 and main; uv.lock β†’ 1.29.0), a federated Streamable HTTP upstream that dies after it has sent the text/event-stream response headers for tools/call never resolves the pending request inside the vendored SDK: mcp/client/streamable_http.py::_handle_sse_response (1.29.0 :429-435) logs the stream EOF at DEBUG and returns without writing to ctx.read_stream_writer when no event id was seen (_handle_reconnection's give-up at :446-448 has the same shape). tool_service.py then waits out anyio.fail_after(effective_timeout) (the session.call_tool frames β€” :6052/:6100/:6241/:6294 in our built v1.0.8 image) and the caller gets the timeout rendering with builtins.TimeoutError as the class. Measured with TOOL_TIMEOUT=5: upstream SIGKILLed 1.08 s into a 4 s call β†’ error returned after 5.07 s. At the default 60 s budget that is ~59 s of dead air, and the caller cannot tell a dead upstream from a slow one.

    The fix exists upstream in the mcp SDK on 2.x β€” modelcontextprotocol/python-sdk#3047 (_resolve_abandoned_request, shipped in mcp 2.0.0) resolves the abandoned request with a CONNECTION_CLOSED JSON-RPC error at exactly those two sites β€” but is absent from 1.29.0/1.29.1 and the v1.x branch, so the <2 pin blocks it. Backport request filed: modelcontextprotocol/python-sdk#3441.

    Two possible closures on the ContextForge side:

    1. Move to mcp>=2 (which also carries the SSE-EOF resolution), or
    2. In the interim, vendor the two-site resolution (the same messages and code=CONNECTION_CLOSED) β€” that is what we are carrying as a build-time patch on the vendored SDK file in our image. With it in place the same fault renders in ~1.1 s as Tool invocation failed: SSE stream ended without a response (httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read)) (mcp.shared.exceptions.McpError) (the class suffix is our earlier str(root_cause)-fallback variant from this issue, now applied uniformly).

    Note for the record on the original report above: the shipped variant of our patch now appends (<module>.<qualname>) on every transport fault rather than only when str(root_cause) is empty β€” httpx.ConnectError carries prose (All connection attempts failed) and was otherwise left untyped.

  2. changed the title [-][BUG]: Federated tool errors from the anyio/httpx transport family surface as an empty "Tool invocation failed: " (str(root_cause) is ''); ExceptionGroup-wrapped timeouts bypass the typed timeout lanes[/-] [+]Federated tool errors from the anyio/httpx transport family surface as an empty "Tool invocation failed: " (str(root_cause) is ''); ExceptionGroup-wrapped timeouts bypass the typed timeout lanes[/+] on Sep 8, 2026
  3. added a commit that references this issue on Sep 28, 2026
    833b94b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions