Skip to content

[Feature] Managed azure-wi-webhook-controller-manager should enforce replica spread across nodes to prevent fail-closed admission failures #5865

Description

@tdjustus

Is your feature request related to a problem? Please describe.

The AKS-managed Azure Workload Identity mutating admission webhook (azure-wi-webhook-controller-manager, deployed in kube-system) runs multiple replicas with a nodeAffinity preference for the system node pool, but no podAntiAffinity or topologySpreadConstraints. Without a spreading constraint, replicas will co-locate on a single node. This design is prone to intermittent admission timeouts that result in pod creation failures for all workload-identity pods during transient node pressure events.

The MutatingWebhookConfiguration (azure-wi-webhook-mutating-webhook-configuration) is configured with failurePolicy: Fail, timeoutSeconds: 10, and objectSelector: azure.workload.identity/use=true. This means transient network or CPU pressure on that single node will simultaneously degrade all webhook replicas and cause admission failures for every workload-identity pod, with no fallback.

Because this Deployment is fully AKS-managed and reconciled by the managed addon, customers cannot apply scheduling constraints durably. This is confirmed by two labels on the Deployment: kubernetes.azure.com/managedby: aks and Flux HelmRelease annotation helm.toolkit.fluxcd.io/name: workload-identity-adapter-helmrelease. Any manual edit to replicas, anti-affinity, resources, or timeout is overwritten on the next reconciliation cycle.

Describe the solution you'd like

Please add a topologySpreadConstraints entry to the managed Deployment spec with topologyKey: kubernetes.io/hostname and maxSkew: 1, so that replicas are scheduled across different nodes.

Goal: a single node's transient network or CPU pressure can no longer degrade all webhook replicas simultaneously.

Describe alternatives you've considered

A requiredDuringSchedulingIgnoredDuringExecution pod anti-affinity rule on kubernetes.io/hostname would also achieve replica spreading and is an acceptable alternative to topologySpreadConstraints. Either approach resolves the co-location risk.

Manual edits to the Deployment are not a durable solution because the managed addon reconciles them back on the next cycle. There is no supported mechanism for customers to patch AKS-managed Deployments persistently.

Additional context

For additional resilience in churn-heavy clusters, the following are also requested for Microsoft's consideration:

  1. Raise timeoutSeconds from 10 to 30 to provide more tolerance for brief apiserver-to-webhook latency.
  2. Increase the replica count from 2 to 3 and raise the per-pod CPU limit above its current 200m (current resource profile: requests cpu=100m/mem=20Mi, limits cpu=200m/mem=300Mi), so the webhook has more headroom during pod-creation storms.

Metadata

Metadata

Labels

Scale and PerformanceUse this for any AKS scale or performance related issuefeature-requestRequested Features

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions