Skip to content

Add endpoint-aware agent configuration#15

Draft
larohra with Copilot wants to merge 13 commits into
mainfrom
copilot/add-endpoint-option-to-front-matter
Draft

Add endpoint-aware agent configuration#15
larohra with Copilot wants to merge 13 commits into
mainfrom
copilot/add-endpoint-option-to-front-matter

Conversation

Copilot AI commented May 19, 2026

Copy link
Copy Markdown
Contributor

Front matter only allowed model, which was too narrow for configuring provider-backed agent execution. This adds a provider-agnostic agent configuration surface so endpoint/model/runtime options can be specified consistently at global or per-agent scope.

This pull request introduces a major change to the configuration model for selecting and managing AI model providers in the Azure Functions Agents runtime. The previous approach, which relied on environment variable auto-detection and top-level model fields, is replaced by a unified, explicit agent_configuration block in YAML. This new configuration is now the single source of truth for provider selection, model settings, and authentication, and is merged agent-over-global. The documentation and sample configs are updated accordingly, and legacy environment variables like MAF_MODEL and MAF_PROVIDER are deprecated in favor of this new approach.

Configuration model overhaul:

  • Replaces the legacy top-level model and provider auto-detection with a unified agent_configuration YAML block, which specifies the provider, model, and all relevant settings, and is now the canonical way to configure model providers. [1] [2] [3] [4]
  • Updates documentation in README.md and docs/architecture.md to describe the new configuration approach, including explicit examples and migration notes. [1] [2] [3] [4]

Sample and environment variable updates:

  • Updates all sample agents.config.yaml files to use the new agent_configuration block, and updates sample documentation to reference the required environment variables for the new config structure. [1] [2] [3] [4]
  • Removes references to deprecated environment variables (MAF_MODEL, MAF_PROVIDER, etc.) and clarifies that secrets and required values should be referenced from agent_configuration. [1] [2] [3]

Code cleanup:

  • Removes DEFAULT_MODEL and DEFAULT_TIMEOUT exports from the package, reflecting the move to configuration-driven defaults. [1] [2]

Internal architecture and extension points:

  • Updates the architecture docs to clarify that agent_configuration is the hand-off contract for execution settings and for custom client managers, replacing the old model-centric approach. [1] [2]

These changes standardize and simplify model provider configuration, improve clarity for users, and pave the way for more robust and extensible agent deployments.

Sample -

name: Azure OpenAI Provider Agent
description: Declares an Azure OpenAI provider block with explicit auth settings.
agent_configuration:
  provider: azure_openai
  temperature: 0.4
  top_p: 0.85
  max_tokens: 512
  azure_openai:
    model: gpt-4.1
    azure_endpoint: https://azure-openai.example.test
    api_version: "2024-10-21"
    api_key: azure-openai-key

Copilot AI linked an issue May 19, 2026 that may be closed by this pull request
Copilot AI changed the title [WIP] Add endpoint option to front-matter specs Add endpoint-aware agent configuration May 19, 2026
Copilot AI requested a review from larohra May 19, 2026 20:29
larohra added 2 commits May 20, 2026 16:03
- Updated test_config_merge.py to utilize agent configuration for OpenAI and Azure providers, ensuring proper precedence and merging behavior.
- Enhanced test_config_validation.py with additional validation checks for agent configurations, including rejection of unknown providers and multiple provider sub-blocks.
- Modified test_registration_endpoints.py to include agent configuration in resolved agents for endpoint tests.
- Adjusted test_registration_handlers.py to ensure agent configuration is passed correctly in HTTP handler tests.
- Implemented test_registration_triggers.py to validate agent configurations in trigger scenarios.
- Introduced test_runner.py to validate agent session history and configuration handling in the runner module.
@larohra
larohra requested a review from Copilot May 21, 2026 17:21
@larohra
larohra removed their request for review May 21, 2026 17:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR replaces the legacy model/provider auto-detection + top-level model/timeout configuration with a unified agent_configuration block (global and per-agent) that becomes the single source of truth for provider selection, universal runtime knobs, and provider-specific settings. It also introduces a provider registry + pluggable client-manager implementation to build Microsoft Agent Framework clients from that configuration, and updates tests/docs/samples accordingly.

Changes:

  • Introduce AgentConfiguration in the config schema and merge it agent-over-global into ResolvedAgent.
  • Add a provider registry and MAF-backed ClientManager that constructs provider clients from agent_configuration (including managed identity vs API key behaviors).
  • Update runner, registration, loader/validation, docs, samples, and fixtures to use agent_configuration and remove legacy top-level fields and env-var fallback behavior.

Reviewed changes

Copilot reviewed 51 out of 51 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
tests/test_runner.py Adds coverage to ensure runner forwards agent_configuration and maps universal knobs into ChatOptions.
tests/test_registration_triggers.py Updates registration trigger tests to supply agent_configuration in fixtures and resolved-agent builders.
tests/test_registration_handlers.py Ensures HTTP/trigger handlers forward agent_configuration into runner calls.
tests/test_registration_endpoints.py Ensures debug endpoints forward agent_configuration into runner calls.
tests/test_config_validation.py Adds validation tests for provider selection and provider sub-block shape rules.
tests/test_config_merge.py Replaces legacy model/timeout precedence tests with agent_configuration merge and precedence tests.
tests/test_config_loader.py Updates loader tests to parse/validate agent_configuration, reject legacy fields, and validate env-substitution behavior.
tests/test_config_fixtures.py Updates fixture-based config scenario assertions to use agent_configuration.
tests/test_client_manager.py Adds comprehensive coverage for provider dispatch, auth mode selection, and secret logging guarantees.
tests/test_app.py Ensures generated test agents include an agent_configuration block when absent.
tests/fixtures/config_scenarios/15_agent_configuration_invalid/unknown_provider.agent.md Adds invalid-provider fixture for loader/validation tests.
tests/fixtures/config_scenarios/15_agent_configuration_invalid/multiple_provider_sub_blocks.agent.md Adds fixture that declares multiple provider sub-blocks to validate rejection.
tests/fixtures/config_scenarios/15_agent_configuration_invalid/credential_extra_passthrough.agent.md Adds fixture that attempts to pass a credential via YAML to validate rejection.
tests/fixtures/config_scenarios/15_agent_configuration_invalid/azure_openai_mutual_exclusivity.agent.md Adds fixture validating Azure OpenAI auth mutual-exclusivity rules.
tests/fixtures/config_scenarios/15_agent_configuration_invalid/agents.config.yaml Adds minimal scenario-level config for invalid-fixture suite.
tests/fixtures/config_scenarios/14_managed_identity_auth/foundry_user_assigned_mi.agent.md Adds managed-identity Foundry example fixture.
tests/fixtures/config_scenarios/14_managed_identity_auth/foundry_system_mi_default.agent.md Adds system-assigned MI Foundry example fixture.
tests/fixtures/config_scenarios/14_managed_identity_auth/azure_openai_user_assigned_mi.agent.md Adds user-assigned MI Azure OpenAI example fixture.
tests/fixtures/config_scenarios/14_managed_identity_auth/azure_openai_system_mi_default.agent.md Adds system-assigned MI Azure OpenAI example fixture.
tests/fixtures/config_scenarios/14_managed_identity_auth/azure_openai_api_key.agent.md Adds API-key Azure OpenAI example fixture.
tests/fixtures/config_scenarios/14_managed_identity_auth/agents.config.yaml Adds minimal scenario-level config for MI fixtures.
tests/fixtures/config_scenarios/13_agent_configuration_providers/openai_agent.agent.md Adds OpenAI provider example fixture.
tests/fixtures/config_scenarios/13_agent_configuration_providers/foundry_agent.agent.md Adds Foundry provider example fixture.
tests/fixtures/config_scenarios/13_agent_configuration_providers/azure_openai_agent.agent.md Adds Azure OpenAI provider example fixture.
tests/fixtures/config_scenarios/13_agent_configuration_providers/agents.config.yaml Adds minimal scenario-level config for provider example fixtures.
tests/fixtures/config_scenarios/12_escaped_placeholders/escaped_literals.agent.md Migrates escaped-placeholder fixture from legacy model to agent_configuration.
tests/fixtures/config_scenarios/10_connector_tools_and_partial_idents/agents.config.yaml Migrates connector-tools scenario config to agent_configuration.
tests/fixtures/config_scenarios/06_capability_filtering/agents.config.yaml Migrates capability-filtering scenario config to agent_configuration.
tests/fixtures/config_scenarios/05_multi_trigger/agents.config.yaml Migrates multi-trigger scenario config to agent_configuration.
tests/fixtures/config_scenarios/04_substitute_variables_false/literal.agent.md Migrates substitute-variables scenario fixture to agent_configuration.
tests/fixtures/config_scenarios/03_env_substitution/azure_reporter.agent.md Removes legacy per-agent model override from env-substitution fixture.
tests/fixtures/config_scenarios/03_env_substitution/agents.config.yaml Migrates env-substitution scenario config to agent_configuration.
tests/fixtures/config_scenarios/02_global_defaults/agents.config.yaml Migrates global-defaults scenario config to agent_configuration.
src/azure_functions_agents/runner.py Switches runner to accept agent_configuration, build ChatOptions from universal knobs, and call the new client manager API.
src/azure_functions_agents/registration/endpoints.py Updates debug execution paths to pass agent_configuration instead of legacy model/timeout.
src/azure_functions_agents/registration/_handlers.py Updates trigger/HTTP handler execution paths to pass agent_configuration.
src/azure_functions_agents/config/validation.py Adds structural agent_configuration validation post-merge and wires it into validate_resolved_agent.
src/azure_functions_agents/config/schema.py Introduces AgentConfiguration and updates GlobalConfig/AgentSpec/ResolvedAgent to use it.
src/azure_functions_agents/config/merge.py Implements agent-over-global composition/merge rules for agent_configuration.
src/azure_functions_agents/config/loader.py Prevents pydantic validation-error chaining to reduce risk of leaking secrets via exception causes.
src/azure_functions_agents/client_manager/providers.py Adds provider registry, per-provider config models, and provider client factories with auth selection.
src/azure_functions_agents/client_manager/init.py Replaces old client_manager.py with a package implementing the new ClientManager API.
src/azure_functions_agents/client_manager.py Removes the legacy client manager that relied on env-var auto-detection and model overrides.
src/azure_functions_agents/init.py Removes exports of DEFAULT_MODEL/DEFAULT_TIMEOUT from the public API surface.
samples/README.md Updates sample guidance to refer to config-driven provider selection and referenced env vars.
samples/daily-tech-news-email/src/agents.config.yaml Migrates sample config to agent_configuration (Azure OpenAI).
samples/daily-azure-report/src/agents.config.yaml Migrates sample config to agent_configuration (Azure OpenAI).
samples/basic-chat/src/agents.config.yaml Migrates sample config to agent_configuration (Azure OpenAI).
README.md Updates docs to describe agent_configuration and custom client-manager integration with the new API surface.
docs/front-matter-spec.md Updates the specification to document agent_configuration as the canonical model/provider config mechanism.
docs/architecture.md Updates architecture notes to reflect agent_configuration handoff and new configuration flow.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/front-matter-spec.md Outdated
| **Global** (`agents.config.yaml`) | None (entire file is optional) | `system_tools`, `model`, `timeout`, `tools` |
| **Agent** (`.agent.md` front matter) | `name`, `description`, `trigger`* | `debug`, `model`, `timeout`, `system_tools`, `mcp`, `skills`, `tools`, `input_schema`, `response_schema`, `response_example`, `metadata` |

| **Global** (`agents.config.yaml`) | None (entire file is optional) | `mcp`, `system_tools`, `agent_configuration`, `tools` |
Comment thread docs/architecture.md Outdated
Comment on lines +215 to +219
@@ -216,7 +216,7 @@ This split keeps parsing, policy, Azure binding registration, and runtime execut

To plug in a different chat backend, implement the `ClientManager` interface and register it once with `set_client_manager(...)`; after that, `runner.run_agent()` and `runner.run_agent_stream()` use your implementation for every call. See `src/azure_functions_agents/client_manager.py` and the README section [Plugging in a custom client manager](../README.md#plugging-in-a-custom-client-manager).

This extension point is deliberately below the registration layer: no trigger or endpoint code needs to change when you swap providers. The `ResolvedAgent.model` value is still the hand-off contract, but your manager decides how to interpret it.
This extension point is deliberately below the registration layer: no trigger or endpoint code needs to change when you swap providers. `ResolvedAgent.agent_configuration` is the hand-off contract, and your manager decides how to interpret it.
model_config = ConfigDict(extra="forbid", populate_by_name=True)

provider: str
timeout: int | None = None
Comment on lines +96 to +100
api_key = kwargs.get("api_key")
explicit_credential = kwargs.get("credential")
managed_identity_client_id = kwargs.pop("managed_identity_client_id", None)
if api_key:
auth_mode = "api_key"
Comment thread src/azure_functions_agents/runner.py Outdated
Comment on lines +353 to +357
timeout = float(agent_configuration.timeout) if agent_configuration.timeout is not None else None

agent, session, resolved_id = await _build_agent_session_history(
instructions=instructions,
agent_configuration=agent_configuration,
Comment thread src/azure_functions_agents/runner.py Outdated
Comment on lines 465 to 469
timeout = float(agent_configuration.timeout) if agent_configuration.timeout is not None else None

try:
agent, session, resolved_id = await _build_agent_session_history(
instructions=instructions,
larohra added 5 commits May 21, 2026 13:11
…nt configuration

- Implement tests to ensure that missing `azure_endpoint` in `azure_openai` and
  missing `project_endpoint` in `foundry` raise appropriate validation errors.
- Update agent configuration handling to accept both top-level and sub-block models.
- Enhance error messages for clarity and to avoid leaking sensitive information.
- Refactor existing tests to improve coverage and ensure correct behavior of agent
  configuration merging and validation.
@anthonychu

Copy link
Copy Markdown
Member

I think the “provider sub-block model wins over top-level model” behavior is a sign that the schema may be too ambiguous. I’d strongly prefer that model only exist in one place.

Instead of allowing both:

agent_configuration:
  model: gpt-4.1
  azure_openai:
    model: gpt-4o-mini

could we make model a single canonical top-level field and keep the provider block focused on provider identity/connection settings? For example:

agent_configuration:
  model: gpt-4.1
  temperature: 0.2
  top_p: 0.95
  max_tokens: 1000
  timeout: 900
  provider:
    name: azure_openai
    endpoint: $AZURE_OPENAI_ENDPOINT
    api_version: "2024-10-21"
    managed_identity_client_id: $AZURE_CLIENT_ID

That avoids needing precedence rules like “provider sub-block model wins,” and avoids the surprising workaround where an agent has to set:

agent_configuration:
  model: gpt-4.1
  azure_openai:
    model: null

In short: I don’t think we should support model in two places. It makes overrides harder to reason about and creates surprising merge behavior.

larohra added 2 commits May 26, 2026 08:39
- Updated tests to replace direct access to provider-specific model fields with a unified top-level model field in agent configurations.
- Adjusted assertions and validation logic across multiple test files to ensure compatibility with the new model structure.
- Removed redundant model fields from provider sub-blocks, simplifying the configuration schema.
- Enhanced validation to reject configurations with empty or missing model fields, ensuring stricter adherence to expected configurations.
…sample configurations

- Updated Python version in GitHub Actions and sample configurations from 3.12 to 3.13.
- Added new timeout handling logic in the agent runner to enforce per-agent-run wall-clock deadlines.
- Updated README and documentation to reflect changes in timeout behavior and configuration.
- Modified sample requirements to use a development requirements file for local development.
- Introduced prepackage hooks in Azure YAML files to build local runtime wheels before packaging.
- Enhanced tests to cover new timeout behavior and ensure correct handling of timeout scenarios.
…option-to-front-matter

# Conflicts:
#	CONTRIBUTING.md
#	README.md
#	docs/architecture.md
#	docs/front-matter-spec.md
#	samples/README.md
#	samples/basic-chat/README.md
#	samples/basic-chat/src/agents.config.yaml
#	samples/basic-chat/src/requirements.txt
#	samples/daily-azure-report/README.md
#	samples/daily-azure-report/src/agents.config.yaml
#	samples/daily-azure-report/src/mcp.json
#	samples/daily-azure-report/src/requirements.txt
#	samples/daily-tech-news-email/README.md
#	samples/daily-tech-news-email/src/agents.config.yaml
#	samples/daily-tech-news-email/src/requirements.txt
#	src/azure_functions_agents/__init__.py
#	src/azure_functions_agents/client_manager.py
#	src/azure_functions_agents/config/validation.py
#	src/azure_functions_agents/discovery/mcp.py
#	src/azure_functions_agents/runner.py
#	tests/fixtures/config_scenarios/10_connector_tools_and_partial_idents/agents.config.yaml
#	tests/test_config_fixtures.py
#	tests/test_config_merge.py
#	tests/test_config_schema.py
#	tests/test_config_validation.py
#	tests/test_connectors_arm_credential.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Endpoint option to front-matter.specs

4 participants