feat(operator): add administrative provider drain

[nerdalert]

Summary

Adds reversible administrative draining for individual inference providers and for all providers explicitly associated with a provider gateway.

A drained provider remains in the routing overlay as existing_only: existing affinity sessions may continue using it, while new sessions are placed elsewhere. Health, trust, freshness, and exclusion decisions continue to take precedence, so draining or undraining cannot make an otherwise ineligible provider routable.

Changes

• Adds InferenceProvider.spec.trafficPolicy.drain to the API, generated CRDs, and Helm schema/template.
• Adds optional InferenceProvider.spec.gatewayRef administrative grouping metadata.
• Applies administrative drain after the operator's existing health and eligibility evaluation.
• Adds cargo xtask env provider-drain with mutually exclusive --provider and --gateway selectors, dry-run support, bounded kubectl operations, and optional accepted/serving revision convergence checks.
• Restores providers after qualification failures and rolls back providers already changed when gateway-wide fan-out fails partway through.
• Extends the single-cluster multi-gateway qualification with individual drain, gateway-wide drain, affinity retention, new-session exclusion, operator restart persistence, unhealthy-provider precedence, invalid-selector handling, and restoration evidence.
• Documents the drain contract and operational workflow.

gatewayRef is grouping metadata used by the administrative command. Gateway-wide drain is an explicit client-side fan-out; it does not add request-time Grid behavior or infer membership from endpoint URLs.

Validation

Static validation passed:

• cargo test --workspace: 561 passed
• cargo clippy --workspace --all-targets -- -D warnings
• make test
• make doc
• make lint
• nightly formatting check
• git diff --check
• Forge topology validation
• focused provider-drain tests: 23 passed

Fresh Kind validation from the final source passed twice:

• Run 2: 32/32 scenarios, automatic restoration and teardown passed
• Run 3: 32/32 scenarios, automatic restoration and teardown passed

The initial attempt used the mock-provider image for the VCR backend and failed before scenarios; its automatic cleanup passed. The two corrected runs used fresh role-appropriate images and completed successfully.

No AI or Praxis changes are included.

[praxis-bot]

praxis-bot review

Well-structured feature. The operator-side drain logic is clean and correct: apply_administrative_drain is pure, only caps NewAndExisting to ExistingOnly, and never overrides hard exclusions. CRD, Helm, and deploy artifacts are consistent. The RestorationGuard with Drop-based cleanup and the bounded retry/rollback in the xtask are solid operational safety patterns. Test coverage is thorough across unit, CRD round-trip, and e2e qualification scenarios.

Two medium findings below on the overlay lookup in the xtask convergence path.