Skip to content

Add read-only preflight for existing AKS Foundry hosting - #10864

Open
Yi Liu (yiliuTo) wants to merge 9 commits into
Azure:mainfrom
yiliuTo:yiliuto-byo-aks-without-operator-preflight
Open

Yi Liu (yiliuTo) wants to merge 9 commits into
Azure:mainfrom
yiliuTo:yiliuto-byo-aks-without-operator-preflight

Conversation

@yiliuTo

@yiliuTo Yi Liu (yiliuTo) commented Nov 6, 2026 •

Copy link
Copy Markdown
Member

Fixes #10769.

Summary

  • Run read-only preflight before Foundry ARM deployments and previews using an existing AKS cluster. Validate supplied resource existence and identity metadata, subscription owning tenants, supplied-identity grants, AKS hosting prerequisites, and the selected existing Agent subnet. Read default dependency existence only when a supplied identity's grant check requires it; default hosting-management identities do not require existence queries.
  • Require running Kubernetes >= 1.31, supported Azure CNI and network policy, managed cluster/kubelet identity metadata, OIDC and Workload Identity, a System pool, and one User pool with Azure Linux, Kata isolation, a compatible node image, and the sandbox label. Recommend System pool capacity of 4 available vCPU / 16 GiB and Agent pool capacity of 8 available vCPU / 32 GiB through non-blocking warnings.
  • Validate supplied Storage provisioning state, StorageV2 kind, Hot tier, HNS disabled, HTTPS-only transport, and TLS >= 1.2. Report zone redundancy as a non-blocking recommendation.
  • Derive the preflight plan from actual compiled ARM templates and merged parameters, including customized on-disk Bicep, nested deployments, and cross-subscription resource scopes. Warn and skip only checks with unresolved local dependencies; failed or unverified mandatory Azure evidence blocks deployment and preview.
  • Reuse preflight identity and kubelet facts for deployment bindings and share role-name resolution. Preserve explicit parameter bindings, ARM parameter envelopes, optional supporting-resource selections, registry lifecycle, and ordinary provisioning/teardown behavior.
  • Keep documentation and existing coverage aligned with the preflight contract.

Validation

  • Projects extension builds successfully, including the Windows CI PIE, coverage, and release-tag build configuration.
  • The full existing Projects test suite passes. Existing native Bicep ACR-mode compile cases also pass with the supported local tool on PATH.
  • The standalone synthesis test suite passes. Unconfigured opt-in integration checks remain skipped; onboarding and provisioning have no skipped tests.
  • Extension spelling checks pass across 983 files. gofmt, targeted go fix -diff, whitespace, and Go line-length checks pass.
  • Validation is offline; no live Azure provisioning, runtime connectivity checks, or full repository preflight were performed.

Related: #10824.
Supersedes #10828.

Yi Liu (yiliuTo) and others added 5 commits November 3, 2026 22:41
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve registry lifecycle validation, process feature gates, and parallel binding reads alongside read-only preflight.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove operator authorization checks and narrow AKS and Storage prerequisites. Report pool capacity as non-blocking recommendations and align documentation and existing tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Adapt preflight provisioning to the shared synthesis module and isolate existing offline fixtures from monitoring defaults.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical preflight flaws can miss deny assignments, bypass required checks, or validate the wrong resources, allowing deployment to proceed without reliable resource and authorization evidence.

11 open issues
What changed in this PR

Adds read-only preflight to the Foundry provisioning provider so existing-AKS deployments and previews validate resource, cluster, network, and identity-access prerequisites before ARM execution.

Changes:

  • Adds resource and authorization checks with blocking failures and advisory warnings.
  • Derives selections from compiled ARM templates and reuses preflight facts for deployment bindings.
  • Updates documentation and test coverage, and shares Bicep test-tool discovery.

Extension user-contract review: complete (35 extension files inspected; 0 new contract comments; 0 existing threads).

Review was static; no builds or tests were executed.

File Description
docs/​reference/​feature-status.md Updated as part of this pull request.
docs/​guides/​creating-an-extension.md Updated as part of this pull request.
docs/​architecture/​extension-framework.md Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​README.md Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​testutil/​bicep.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​synthesis/​templates_embed.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​synthesis/​schema_test.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​synthesis/​hosting_test.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​provisioning/​supplied_resource_bindings.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​provisioning/​onboarding_naming.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​provisioning/​onboarding_naming_test.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​provisioning/​foundry_provisioning_provider.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​provisioning/​byo_template_plan.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​provisioning/​byo_template_plan_test.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​provisioning/​byo_plan_expression.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​provisioning/​byo_plan_bindings.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​provisioning/​aks_hosting.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​provisioning/​aks_hosting_compatibility_test.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​onboarding/​types.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​onboarding/​resource_test.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​onboarding/​resource_sdk.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​onboarding/​resource_rules.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​onboarding/​resource_clients.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​onboarding/​plan.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​onboarding/​pipeline.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​onboarding/​parallel_reads.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​onboarding/​network_sdk.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​onboarding/​network_configuration.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​onboarding/​managed_node_network.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​onboarding/​identity_authorization.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​onboarding/​authorization_test.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​onboarding/​authorization_sdk.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​onboarding/​authorization_rules.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​internal/​onboarding/​authorization_evidence.go Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​go.sum Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.projects/​go.mod Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.agents/​README.md Updated as part of this pull request.
cli/​azd/​extensions/​azure.ai.agents/​internal/​synthesis/​templates_embed.go Updated as part of this pull request.
cli/​azd/​.vscode/​cspell.yaml Updated as part of this pull request.

Review effort: Max

Comment on lines +119 to +121
permissions = append(permissions, role.Properties.Permissions...)
if roleCovers(permissions, required.Properties.Permissions) {
return Pass, "verified supplied-identity permission allow at an applicable scope", nil
Comment on lines +467 to +471
if !resource.Supplied {
return Identity{}, new(true), resourceKnown(check, response.ID != nil,
strings.EqualFold(authString(response.ID), resource.ID),
"default identity exists; its ordinary ARM creation/update path is retained")
}
kubeletObjectID, err = p.readBYOKubeletObjectID(readCtx, clusterID)
if err != nil {
cancelReads(err)
plan, err := p.identifyBYOTemplatePlan(ctx, source)
Comment on lines +161 to +163
if !strings.EqualFold(identity.id, identity.fact.ID) || identity.fact.PrincipalID == "" {
return fmt.Errorf("%s deployment binding requires matching resource ID and principal ID from preflight",
identity.field)
Comment on lines +130 to +132
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
r.err = errors.Join(r.err, err)
}
Comment on lines +447 to +450
actual, why := node.id()
if why != "" {
uncertain = node.path + ": " + why
continue
Comment on lines +89 to +90
message.WriteString(fmt.Sprintf("; %s %s: %s (%s)",
c.ID, c.Status, RedactDiagnostic(c.Observed), RedactDiagnostic(c.Remediation)))
Comment on lines +189 to +190
if output.Len() > 0 {
progress(output.String())
Comment on lines +53 to +54
// BuildPlan independently selects each reference or default without Azure reads or writes.
func BuildPlan(in Input) (Plan, error) {
Comment thread docs/reference/feature-status.md Outdated
@@ -53,6 +53,7 @@ Current maturity status of Azure Developer CLI features. See [Feature Stages](..
| Azure Functions | Stable |
| Azure Kubernetes Service (AKS) | Beta |
| Azure AI | Beta |
| [Microsoft Foundry on existing AKS](../../cli/azd/extensions/azure.ai.projects/README.md#existing-aks-hosting) (default or supplied supporting resources, read-only preflight) | In development (infrastructure enabled by default; Agents init/adoption disabled by default) |
Copilot AI max review requested due to automatic review settings November 6, 2026 13:36
Restore shared documentation and spelling configuration to upstream main, using file-local cspell directives for preflight terminology.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +45 to +46
scope := "/subscriptions/" + clusterID.SubscriptionID + "/resourceGroups/" + nodeRG
groups, err := armresources.NewResourceGroupsClient(clusterID.SubscriptionID, c.credential, c.options)
Copilot AI max review requested due to automatic review settings November 6, 2026 13:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Preflight validation skips hard-coded or additional AKS hosting configurations, allowing deployment or preview to proceed without validating all active resources and required grants.

13 open issues
2 previously missed issues

In code that hasn't changed since last review

High severity Preflight ignores additional Foundry accounts with AKS hosting

cli/​azd/​extensions/​azure.ai.projects/​internal/​provisioning/​byo_plan_bindings.go:57

[P1] If a customized template deploys two Foundry accounts with AKS hosting, selecting the account named by the output silently discards the other account even though ARM will deploy both. An invalid cluster, Storage account, or missing grant on that second account therefore never blocks deployment or preview. Preflight every active AKS hosting configuration in the submitted template, or block when the additional configurations cannot be checked.

Medium severity Treat missing HNS state as disabled

cli/​azd/​extensions/​azure.ai.projects/​internal/​onboarding/​resource_rules.go:348

This blocks common supplied Storage accounts, and the user cannot fix it.

  • Why: isHnsEnabled can only be set when the account is created. Storage returns it as null for accounts created without setting it, for example az storage account create without --hns. Requiring IsHnsEnabled != nil makes storage-type UNVERIFIED, which is mandatory and blocks both deploy and preview.
  • Why it's not fixable: HNS cannot later be explicitly set to false, so the remediation ("ask the resource administrator to correct the configuration") is not possible. The customer would have to recreate the account.
  • Fix: The API default is HNS disabled. That is the same documented-default reasoning this function already applies to HTTPS and TLS. Treat nil as disabled, and fail only on an explicit true (which the check at line 352 already catches).

The HNS missing test case and the README's "HNS explicitly disabled" wording would need matching updates.

Review effort: Max

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI max review requested due to automatic review settings November 6, 2026 14:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI max review requested due to automatic review settings November 7, 2026 08:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if missing != "" {
return planUnknown{missing}
}
properties := map[string]any{}
Comment on lines +152 to +153
// ExistingRoleAssignmentKeys supplies exact scope/principal/role keys for mixed-resource deployment deduplication.
func (e *AuthorizationEvidence) ExistingRoleAssignmentKeys(ctx context.Context, plan Plan) ([]string, Report, error) {
Comment on lines +173 to +175
p := assignment.Properties
keys = append(keys, strings.ToLower(authString(p.Scope)+"|"+authString(p.PrincipalID)+"|"+
roleGUID(authString(p.RoleDefinitionID))))
Comment on lines +13 to +14
if !plan.Hosting.Supplied || !plan.Workload.Supplied || !plan.Storage.Supplied {
return nil
Comment on lines +218 to +222
if plan.Storage.Supplied || plan.Workload.Supplied {
tasks = append(tasks, func() {
storage, existence[1], storageReport = c.readStorage(ctx, plan.Storage)
})
}
readerRoleId
kubernetesExtensionContributorRoleId
]: if (empty(hostingIdentityResourceId)) {
]: if (createRoleAssignments) {
@azure-sdk-automation

Copy link
Copy Markdown
Contributor

azure.ai.agents PR build

Note

This is an unsigned development build. Install it only if you trust this PR.

Install the extension:

azd ext install "https://azuresdkartifacts.z5.web.core.windows.net/azd/extensions/pr/10864/azure-ai-agents.zip"
  • Version: 1.0.0-pr.10864.7050536
  • Merge commit: 0a5c440

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ext-agents azure.ai.agents extension ext-projects azure.ai.projects extension

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BYO AKS stack 03] Add preflight checks for customer-provided resources

2 participants