Repository navigation
azd foundry app insights VNET support - #10819
Kayla Seager (kseager) wants to merge 16 commits into
Conversation
|
Azure Pipelines: 5 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
|
Thank you for your contribution Kayla Seager (@kseager)! We will review the pull request and get back to you soon. |
2152de9 to
8ad9393
Compare
There was a problem hiding this comment.
🟡 Changes recommended
Critical gaps in private-monitoring conflict checks can disrupt shared DNS and existing telemetry, while workspace ingestion remains publicly accessible despite the private-only configuration.
9 open issues
Skip monitoring when endpoint conflict checks are inconclusive · New Check conflicts before the existing-connection early return · New Exempt only the exact planned scope ID from conflict checks · New Gate monitoring creation on BYO egress · New Protect the Bicep deployment path from endpoint conflicts · New Validate shared DNS zones before binding Monitor endpoints · New Disable public ingestion on the workspace too · New Make skip warnings persistent and visible in automated runs · New Document VNet-wide Monitor routing impact and opt-out · New
What changed in this PR
Extends the Foundry infrastructure synthesis shared by the Projects and Agents extensions to support private Application Insights ingestion for new projects using customer-provided agent subnets.
Changes:
- Adds Azure Monitor private-link infrastructure, DNS integration, a project capability host, and an ordered component re-sync.
- Enables BYO-network monitoring with endpoint-conflict checks and private-ingestion policy validation.
- Updates embedded ARM templates, Bicep exports, tests, and networking documentation.
Extension user-contract review: blocked (23 files inspected; 0 new contract comments; 0 matched threads; review history unavailable).
| File | Description |
|---|---|
cli/azd/extensions/azure.ai.projects/synthesis/vnet_requirements_test.go |
Checks capability-host template requirements. |
cli/azd/extensions/azure.ai.projects/synthesis/templates/modules/resources.bicep |
Adds the project capability host and network outputs. |
cli/azd/extensions/azure.ai.projects/synthesis/templates/modules/monitoring.bicep |
Wires private monitoring and component re-sync. |
cli/azd/extensions/azure.ai.projects/synthesis/templates/modules/monitoring-resync.bicep |
Reapplies the component after private-link setup. |
cli/azd/extensions/azure.ai.projects/synthesis/templates/modules/monitor-private-link.bicep |
Defines the Monitor scope, endpoint, and DNS resources. |
cli/azd/extensions/azure.ai.projects/synthesis/templates/main.bicep |
Passes network settings into monitoring. |
cli/azd/extensions/azure.ai.projects/synthesis/templates/main.arm.json |
Regenerates embedded new-project infrastructure. |
cli/azd/extensions/azure.ai.projects/synthesis/templates/existing-project.arm.json |
Regenerates the embedded monitoring module. |
cli/azd/extensions/azure.ai.projects/synthesis/synthesizer.go |
Enables monitoring creation for BYO networks. |
cli/azd/extensions/azure.ai.projects/synthesis/monitoring_test.go |
Updates network-specific monitoring expectations. |
cli/azd/extensions/azure.ai.projects/synthesis/monitoring_resolver.go |
Applies private-network component validation. |
cli/azd/extensions/azure.ai.projects/synthesis/monitoring_private_link.go |
Adds Monitor endpoint-conflict discovery. |
cli/azd/extensions/azure.ai.projects/synthesis/monitoring_private_link_test.go |
Tests conflicts, template settings, and ingestion policy. |
cli/azd/extensions/azure.ai.projects/synthesis/monitoring_policy.go |
Allows disabled public ingestion on private paths. |
cli/azd/extensions/azure.ai.projects/README.md |
Documents private monitoring behavior and limits. |
cli/azd/extensions/azure.ai.projects/internal/provisioning/monitoring_preflight.go |
Integrates private-network monitoring checks and skips. |
cli/azd/extensions/azure.ai.projects/internal/provisioning/monitoring_preflight_test.go |
Retargets the unsupported-network skip test to reuse. |
cli/azd/extensions/azure.ai.projects/internal/provisioning/monitoring_graph.go |
Applies private-network policy during graph validation. |
cli/azd/extensions/azure.ai.projects/internal/cmd/project_existing_infra.go |
Exports the new monitoring Bicep dependencies. |
cli/azd/extensions/azure.ai.projects/cspell.yaml |
Adds Monitor terminology to spell checking. |
cli/azd/extensions/azure.ai.agents/internal/cmd/init_infra.go |
Includes both new modules in Bicep exports. |
cli/azd/extensions/azure.ai.agents/docs/private-networking.md |
Describes private monitoring support and exclusions. |
cli/azd/extensions/azure.ai.agents/cspell.yaml |
Adds the Azure Monitor group identifier. |
Review effort: Max
Adds AMPLS, azuremonitor private endpoint, private DNS zones, project capability host, and a deterministic App Insights component re-sync after the AMPLS scoped resources so ingestion is reliable on fresh environments. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491
… ingestion, docs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491
9257128 to
ee24b66
Compare
There was a problem hiding this comment.
🔵 Needs a closer look
Human assessment is needed to evaluate interacting template-compatibility, network-preflight, and shared-DNS risks that could prevent trace ingestion or disrupt existing environments.
No open issues 🎉
1 resolved since last review
Review effort: Max
Add an ampls-monitor-rule outbound rule to the managed network, grant the account identity the Network Connection Approver role, and stop skipping monitoring for managed egress. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491
There was a problem hiding this comment.
🟡 Changes recommended
The role-assignment GUID omits the principal ID, so recreating the account with the same project ID reuses an assignment name for a different identity and fails with RoleAssignmentUpdateNotPermitted.
1 open issue
Review effort: Max
…name Deploy the assignment through a module so the GUID is derived from scope, principal and role, avoiding RoleAssignmentUpdateNotPermitted when the account is recreated with the same project ID. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491
There was a problem hiding this comment.
🟡 Changes recommended
Existing-project exports omit a required Bicep module, causing compilation failures, and unresolved vnetId values can silently drop an existing monitoring ownership binding.
2 open issues
1 resolved since last review
Review effort: Max
monitoring.bicep references the module, so both eject paths must copy it or compilation fails. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491
Hosted agents fail with 'Public access is disabled' under a managed network unless the account has an outbound private endpoint rule to itself. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491
There was a problem hiding this comment.
🔵 Needs a closer look
Human assessment is needed because unpublished findings identify critical VNet and DNS validation gaps that could bypass network safeguards or redirect another VNet’s telemetry to the wrong endpoint.
No open issues 🎉
2 resolved since last review
Review effort: Max
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491
There was a problem hiding this comment.
🔵 Needs a closer look
Human assessment is needed because private monitoring can be approved without a verified network path, with additional connectivity and ownership risks across configuration changes.
No open issues 🎉
1 resolved since last review
Review effort: Max



Scope
• It creates the AMPLS and PE and adds the ampls-monitor-rule outbound rule.
• It grants the account identity the Network Connection Approver role.
• It also adds the foundry-account-pe outbound rule (account to itself), required for hosted agents.
• If no isolation mode is set, it defaults to AllowInternetOutbound .
• This covers connecting an existing App Insights component. It creates no new component for an existing project.
• Existing-project AMPLS, PE and rule creation stays out of scope and is documented
Problem
A project with a
network:block (VNet-injected agents) could not send traces: native azd monitoring created public-ingestion App Insights, and #10743 skips monitoring in that case. This PR adds a private path so agent traces arrive in App Insights and can be read from inside the network.What azd now provisions (new project,
network:with your own agent subnet, monitoring enabled)Ingestion is private only. Traces are sent through the AMPLS private endpoint and the component rejects public ingestion. Query matches Foundry sample 15 (AMPLS query
Open). Difference with S2 bug bash sample for VNET is that our set-up connects the App Insight/workspace to the project vs Foundry.Component re-sync (reliability fix)
In earlier fresh environments, ingestion intermittently returned
403 Component private link id does not match incoming linkfor 45 minutes or more (one environment stayed stuck for hours) after ARM reported everything Succeeded. Touching the component's tags cleared it immediately.Suspected cause: a stale routing cache. Private Link data-plane traffic is handled by internal distributed proxies (the Private Link mapping layer), which ARM does not expose. When the component is created, it registers routing metadata with no AMPLS link. Linking it through the AMPLS scoped resource updates the AMPLS side (the
scopedServicestable), but the proxy mapping for the component appears to be treated as a cache that is only refreshed by a modification event on the component itself. Until one arrives, the mapping can stay stale and ingestion keeps being rejected as a link-id mismatch. ARM only sees control-plane state, so it reports Succeeded while the data plane is still stale.A second PUT on the component, even one that only changes a tag, makes ARM send a metadata sync to the underlying resource provider. That should invalidate the stale mapping and reload the link ID.
This mechanism is inferred from the symptoms. It is not confirmed by Azure Monitor or Private Link documentation, and I have not verified it with the service team. The only evidence is the live runs below.
To fix this issue reliably
monitoring-resync.bicepre-declares the component with identical properties after the AMPLS scoped resources exist, adding a deterministic tagazd-ampls-link(a hash of the scope ID and component name; noutcNow, so re-provision is a no-op). The project connection depends on it.Test evidence (westus3, jump VM inside the VNet, public ingestion off)
Flow per environment:
azd provisionfrom a laptop;azd deployandazd ai agent invokefrom the jump VM; traces read privately from the VM.Notes and limits:
Behavior and limits
network.dns), a VNet that already has anazuremonitorprivate endpoint, or a network azd cannot inspect do not block provisioning: monitoring is skipped with a warning (opt-out:monitoring.enabled: false).ampls-monitor-rule(targetazuremonitor) lets the agent runtime reach the Private Link Scope.foundry-account-pe(targetaccount, to the account itself) is required for hosted agents. Without it they fail withPublic access is disabled. Prompt agents are not affected.Monitoring Metrics Publisheron the Application Insights component. azd does not grant it yet. In the live run it was granted by hand, so hosted-agent traces can return 403 until it is granted.PrivateOnly, telemetry from the VNet to Azure Monitor resources that are not in the scope is blocked.Open, queries are not restricted to the private network. Verified live: private ingestion plus a read from inside the VNet, not a private-only read.Changes
synthesis/templates/modules/monitor-private-link.bicep: AMPLS, scoped resources,azuremonitorprivate endpoint, four DNS zones (create-and-link or reuse existing).synthesis/templates/modules/monitoring-resync.bicep(new), wired intomonitoring.bicep; added to both eject module lists.resources.bicep.main.arm.jsonandexisting-project.arm.json.TestMonitorPrivateLinkAccessModes,TestMonitoringResyncMatchesComponent,TestCheckMonitorPrivateLinkConflict,TestMonitoringKeyPolicyForPrivateNetwork, VNet requirement tests, preflight tests.README.mdandazure.ai.agents/docs/private-networking.md.Not covered
One region. A fresh managed environment on the final template (including foundry-account-pe) was not provisioned end to end; the managed rules and the agent-identity role were validated by hand on existing environments. Terraform parity and eject for this module follow the same shape but were not live-tested. The project-scoped connection was exercised live only in the runs above.