Skip to content

fix(client_manager): add explicit HTTP timeouts to Foundry chat client (fixes #65)#70

Open
nzthiago wants to merge 2 commits into
Azure:mainfrom
nzthiago:fix/issue-65-foundry-httpx-timeout
Open

fix(client_manager): add explicit HTTP timeouts to Foundry chat client (fixes #65)#70
nzthiago wants to merge 2 commits into
Azure:mainfrom
nzthiago:fix/issue-65-foundry-httpx-timeout

Conversation

@nzthiago

@nzthiago nzthiago commented Jun 15, 2026

Copy link
Copy Markdown
Member

fix(client_manager): add explicit HTTP timeouts to Foundry chat client

Fixes #65

Summary

Configure explicit connect/read HTTP timeouts on the Azure SDK transport
used by the Foundry chat client, so transient Foundry latency surfaces as a
raised exception instead of an indefinite worker-process hang.

Background

Today _build_foundry passes project_endpoint (a string) to
FoundryChatClient, which internally constructs AIProjectClient using the
Azure SDK pipeline's default transport. The default transport has no
httpx.Timeout configured for read operations. When Foundry has any transient
latency on a Responses API roundtrip (POST /openai/v1/responses), the
underlying socket read blocks indefinitely with no exception raised — the agent
loop is suspended forever, no error logs, no traceback. The host's
functionTimeout (30 min on Flex) eventually kills the worker, but by then
the user has lost a full invocation with no way to recover.

Repro is probabilistic and tends to surface after ~10+ Foundry roundtrips in
a single invocation. Issue #65 has the full diagnostic evidence:

  • Live worker-thread stack trace at the hung state (main thread blocked in
    asyncio.run > runner.run > self._loop.run_until_complete(task)).
  • Worker SIGKILL / OOM / segfault ruled out (zero
    WorkerProcessExitException; MemoryWorkingSet peak ~650 MB on Flex
    Consumption Linux).
  • Foundry rate-limit ruled out (5–11% of 900 K TPM, zero 429s).
  • Reproduced across gpt-5, gpt-5.4, gpt-5.4-mini.

Change

In src/azure_functions_agents/client_manager.py, the _build_foundry
classmethod now:

  1. Reads FOUNDRY_HTTP_READ_TIMEOUT (default 180.0 seconds) and
    FOUNDRY_HTTP_CONNECT_TIMEOUT (default 10.0 seconds) from the
    environment.
  2. Constructs an AioHttpTransport with those timeouts.
  3. Constructs AIProjectClient itself with that transport plus the
    existing async credential.
  4. Passes the pre-built project_client to FoundryChatClient (instead of
    the bare project_endpoint string).

This is a no-behavior-change improvement on the happy path; the only
observable difference is that hung roundtrips now raise
httpx.ReadTimeout after the configured window instead of blocking forever,
which lets the agent runtime's existing exception-handling path surface the
failure (and gives users a clean error to retry from).

Defaults

  • FOUNDRY_HTTP_READ_TIMEOUT=180 (3 min) — generous enough for the longest
    legitimate Responses-API call but short enough that a stuck connection
    fails an invocation in roughly 1/6th of the platform timeout window.
  • FOUNDRY_HTTP_CONNECT_TIMEOUT=10 — matches the
    Azure SDK guidance
    for short connection setup windows.

Both are overridable via app settings without touching code.

Out of scope (follow-ups)

The same timeout-gap exists in:

  • _build_openai (lines 153-159)
  • _build_azure_openai (lines 162-186)

OpenAIChatClient accepts a pre-built async_client parameter
(AsyncOpenAI / AsyncAzureOpenAI), both of which take an http_client
parameter where an httpx.AsyncClient(timeout=httpx.Timeout(...)) can be
injected. The same pattern as this PR applies, but the surface is slightly
different (constructor params are split across multiple branches in the
existing code). Happy to send a follow-up PR for those if this lands cleanly.

Test plan

  • Existing tests in tests/test_client_manager.py pass unchanged — the
    resolve-model logic and provider-selection paths are untouched.
  • Manual repro (issue Add Timeout Configuration to Foundry OpenAI Client #65 evidence): with the patched runtime, a 14-query
    Foundry-backed agent invocation that previously hung silently now either
    completes within the read-timeout window or raises httpx.ReadTimeout
    cleanly within ~180 s; the worker process stays responsive and the
    invocation count is accurate in Application Insights.

Risk

Low. The change is local to one classmethod, defaults are conservative and
env-overridable, and the only behavioural difference on the unhappy path is
"raises a documented exception instead of hanging forever".

MCP timeout follow-up

This PR now also fixes the MCP timeout gap in src/azure_functions_agents/discovery/mcp.py.

  • Reads each server's timeout setting from mcp.json (either a scalar or {connect, read, write, pool} object) and applies it to the underlying httpx.AsyncClient transport.
  • Passes the MCP read timeout through to MCPStreamableHTTPTool(request_timeout=...) so stalled tool calls fail cleanly instead of hanging indefinitely.
  • Defaults MCP requests to 120 seconds when no per-server timeout is configured, with MCP_REQUEST_TIMEOUT available as a global override.
  • Always constructs an HTTP client even when no header provider is needed, ensuring every MCP HTTP tool gets timeout protection.

Together with the Foundry transport change, this closes both indefinite-hang paths called out in #65: Foundry model calls and MCP connector/tool calls.

Fixes Azure#65

Configure explicit connect/read HTTP timeouts on the Azure SDK transport
used by the Foundry chat client, so transient Foundry latency surfaces as
a raised exception instead of an indefinite worker-process hang.

Defaults: read=180s, connect=10s. Both overridable via
FOUNDRY_HTTP_READ_TIMEOUT and FOUNDRY_HTTP_CONNECT_TIMEOUT app settings.
@nzthiago

Copy link
Copy Markdown
Member Author

Cross-referencing prior art: microsoft/agent-framework#6263 (merged 2026-06-04) addressed the same class of bug — ConnectTimeout on FoundryAgent multi-turn conversations — by adding a timeout: float | None parameter to FoundryAgent / RawFoundryAgent / _FoundryAgentChatClient / RawFoundryAgentChatClient / RawOpenAIChatClient, threaded down to openai_client.with_options(timeout=...).

That fix only covers the FoundryAgent code path (OpenAI SDK + AsyncAzureOpenAI). Our runtime here uses FoundryChatClient (a different class in the same package), which constructs AIProjectClient rather than going through the OpenAI SDK directly — so it was untouched by #6263. This PR closes the equivalent gap on that side at the runtime layer.

A cleaner long-term fix would be to add the same timeout parameter to FoundryChatClient upstream in agent-framework-foundry (mirroring #6263). If/when that lands, this runtime fix can simplify to a one-liner — FoundryChatClient(project_endpoint=..., model=..., credential=..., timeout=...) — and the explicit AIProjectClient + AioHttpTransport construction here becomes unnecessary. Happy to file that follow-up upstream as well.

…ycle

The MCP server discovery code creates httpx.AsyncClient instances with no
timeout, and never passes request_timeout to MCPStreamableHTTPTool. When
an MCP connector (e.g., Kusto) stalls, the agent hangs indefinitely.

This change:
- Reads the per-server `timeout` config from mcp.json (connect, read,
  write, pool) and applies it to the httpx transport
- Passes `request_timeout` to MCPStreamableHTTPTool for end-to-end
  tool call timeout
- Defaults to 120s when no timeout is configured
- Supports MCP_REQUEST_TIMEOUT env var as a global override

Complements the Foundry client timeout fix in client_manager.py.

Refs: Azure#65

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Timeout Configuration to Foundry OpenAI Client

1 participant