fix(client_manager): add explicit HTTP timeouts to Foundry chat client (fixes #65)#70
fix(client_manager): add explicit HTTP timeouts to Foundry chat client (fixes #65)#70nzthiago wants to merge 2 commits into
Conversation
Fixes Azure#65 Configure explicit connect/read HTTP timeouts on the Azure SDK transport used by the Foundry chat client, so transient Foundry latency surfaces as a raised exception instead of an indefinite worker-process hang. Defaults: read=180s, connect=10s. Both overridable via FOUNDRY_HTTP_READ_TIMEOUT and FOUNDRY_HTTP_CONNECT_TIMEOUT app settings.
|
Cross-referencing prior art: microsoft/agent-framework#6263 (merged 2026-06-04) addressed the same class of bug — That fix only covers the A cleaner long-term fix would be to add the same |
…ycle The MCP server discovery code creates httpx.AsyncClient instances with no timeout, and never passes request_timeout to MCPStreamableHTTPTool. When an MCP connector (e.g., Kusto) stalls, the agent hangs indefinitely. This change: - Reads the per-server `timeout` config from mcp.json (connect, read, write, pool) and applies it to the httpx transport - Passes `request_timeout` to MCPStreamableHTTPTool for end-to-end tool call timeout - Defaults to 120s when no timeout is configured - Supports MCP_REQUEST_TIMEOUT env var as a global override Complements the Foundry client timeout fix in client_manager.py. Refs: Azure#65 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
fix(client_manager): add explicit HTTP timeouts to Foundry chat client
Fixes #65
Summary
Configure explicit
connect/readHTTP timeouts on the Azure SDK transportused by the Foundry chat client, so transient Foundry latency surfaces as a
raised exception instead of an indefinite worker-process hang.
Background
Today
_build_foundrypassesproject_endpoint(a string) toFoundryChatClient, which internally constructsAIProjectClientusing theAzure SDK pipeline's default transport. The default transport has no
httpx.Timeoutconfigured for read operations. When Foundry has any transientlatency on a Responses API roundtrip (
POST /openai/v1/responses), theunderlying socket read blocks indefinitely with no exception raised — the agent
loop is suspended forever, no error logs, no traceback. The host's
functionTimeout(30 min on Flex) eventually kills the worker, but by thenthe user has lost a full invocation with no way to recover.
Repro is probabilistic and tends to surface after ~10+ Foundry roundtrips in
a single invocation. Issue #65 has the full diagnostic evidence:
asyncio.run > runner.run > self._loop.run_until_complete(task)).WorkerProcessExitException;MemoryWorkingSetpeak ~650 MB on FlexConsumption Linux).
gpt-5,gpt-5.4,gpt-5.4-mini.Change
In
src/azure_functions_agents/client_manager.py, the_build_foundryclassmethod now:
FOUNDRY_HTTP_READ_TIMEOUT(default180.0seconds) andFOUNDRY_HTTP_CONNECT_TIMEOUT(default10.0seconds) from theenvironment.
AioHttpTransportwith those timeouts.AIProjectClientitself with that transport plus theexisting async credential.
project_clienttoFoundryChatClient(instead ofthe bare
project_endpointstring).This is a no-behavior-change improvement on the happy path; the only
observable difference is that hung roundtrips now raise
httpx.ReadTimeoutafter the configured window instead of blocking forever,which lets the agent runtime's existing exception-handling path surface the
failure (and gives users a clean error to retry from).
Defaults
FOUNDRY_HTTP_READ_TIMEOUT=180(3 min) — generous enough for the longestlegitimate Responses-API call but short enough that a stuck connection
fails an invocation in roughly 1/6th of the platform timeout window.
FOUNDRY_HTTP_CONNECT_TIMEOUT=10— matches theAzure SDK guidance
for short connection setup windows.
Both are overridable via app settings without touching code.
Out of scope (follow-ups)
The same timeout-gap exists in:
_build_openai(lines 153-159)_build_azure_openai(lines 162-186)OpenAIChatClientaccepts a pre-builtasync_clientparameter(
AsyncOpenAI/AsyncAzureOpenAI), both of which take anhttp_clientparameter where an
httpx.AsyncClient(timeout=httpx.Timeout(...))can beinjected. The same pattern as this PR applies, but the surface is slightly
different (constructor params are split across multiple branches in the
existing code). Happy to send a follow-up PR for those if this lands cleanly.
Test plan
tests/test_client_manager.pypass unchanged — theresolve-model logic and provider-selection paths are untouched.
Foundry-backed agent invocation that previously hung silently now either
completes within the read-timeout window or raises
httpx.ReadTimeoutcleanly within ~180 s; the worker process stays responsive and the
invocation count is accurate in Application Insights.
Risk
Low. The change is local to one classmethod, defaults are conservative and
env-overridable, and the only behavioural difference on the unhappy path is
"raises a documented exception instead of hanging forever".
MCP timeout follow-up
This PR now also fixes the MCP timeout gap in
src/azure_functions_agents/discovery/mcp.py.timeoutsetting frommcp.json(either a scalar or{connect, read, write, pool}object) and applies it to the underlyinghttpx.AsyncClienttransport.MCPStreamableHTTPTool(request_timeout=...)so stalled tool calls fail cleanly instead of hanging indefinitely.MCP_REQUEST_TIMEOUTavailable as a global override.Together with the Foundry transport change, this closes both indefinite-hang paths called out in #65: Foundry model calls and MCP connector/tool calls.