Phase 1: Foundation — add AuthServerConfig model, CRD field, and structural validation

[tgrunnagle]

Description

Add the foundational structural changes required for the vMCP embedded authorization server (RFC-0053, Phase 1). This task introduces all new config model types, CRD fields, and structural validation without changing any runtime behavior — every new field is optional (omitempty) and the vMCP server does not read them yet. Passing all existing tests unchanged is the acceptance gate.

Context

RFC-0053 adds an optional embedded OAuth/OIDC authorization server to vMCP. When Config.AuthServer is nil (Mode A), behavior is byte-for-byte identical to today. When set (Mode B, introduced in Phase 2), the AS acts as the OIDC issuer for incoming clients. This Phase 1 ticket establishes the structural skeleton that Phases 2, 3, and 4 build on: it moves a struct to its canonical home, adds new config and CRD fields, adds a JwksAllowPrivateIP gap-fill needed for Mode B loopback OIDC discovery, and regenerates the deepcopy and CRD manifests.

Parent epic: #4120 — vMCP: add embedded authorization server
RFC document: docs/proposals/THV-0053-vmcp-embedded-authserver.md
Dependencies: None (root task — can start immediately)
Blocks: Phase 2 (server wiring), Phase 3 (startup validation), Phase 4 (operator reconciler)

Acceptance Criteria

• ExternalAuthConfigRef struct is defined only in cmd/thv-operator/api/v1alpha1/mcpexternalauthconfig_types.go and is no longer defined in mcpserver_types.go; all existing callers (v1alpha1.ExternalAuthConfigRef) compile without change
• VirtualMCPServerSpec has a new field AuthServerConfigRef *ExternalAuthConfigRef with +optional marker and json:"authServerConfigRef,omitempty" tag
• VirtualMCPServer.Validate() calls r.validateAuthServerConfig(), which returns an error when AuthServerConfigRef is non-nil and AuthServerConfigRef.Name is empty
• ConditionTypeAuthServerConfigValid = "AuthServerConfigValid" constant is added alongside the existing condition type constants in virtualmcpserver_types.go
• pkg/vmcp/config.Config has a new field AuthServer *AuthServerConfig with json:"authServer,omitempty" yaml:"authServer,omitempty" and a // +optional comment
• pkg/vmcp/config.AuthServerConfig struct wraps *authserver.RunConfig with +kubebuilder:object:generate=true annotation
• pkg/vmcp/config.OIDCConfig has a new JwksAllowPrivateIP bool field with json:"jwksAllowPrivateIP,omitempty" yaml:"jwksAllowPrivateIP,omitempty" tag
• JwksAllowPrivateIP is wired through pkg/vmcp/auth/factory/incoming.go's newOIDCAuthMiddleware to auth.TokenValidatorConfig.AllowPrivateIP (note: this field currently maps ProtectedResourceAllowPrivateIP; JwksAllowPrivateIP must map to the same underlying AllowPrivateIP field — confirm the correct mapping with the auth package)
• pkg/vmcp/config/zz_generated.deepcopy.go is regenerated and includes DeepCopyInto/DeepCopy for AuthServerConfig
• task operator-generate runs without error (regenerates cmd/thv-operator/api/v1alpha1/zz_generated.deepcopy.go)
• task operator-manifests runs without error (regenerates CRD YAML in config/crd/bases/)
• task crdref-gen runs without error (run from inside cmd/thv-operator/)
• All existing unit tests pass (task test)
• task lint passes (or task lint-fix produces no unresolvable issues)
• All new Go files include the SPDX license header

Technical Approach

Recommended Implementation

Work in four discrete, independently-committable steps: (1) move ExternalAuthConfigRef in the operator CRD types, (2) add CRD field and validation to VirtualMCPServerSpec, (3) add AuthServerConfig and JwksAllowPrivateIP to the config model and wire the new field through the incoming auth factory, (4) regenerate all generated files. None of these steps changes any runtime behavior because all new fields are guarded by nil checks that Phase 2 introduces.

Patterns and Frameworks

• omitempty for all optional fields: Match the convention used by every optional field in pkg/vmcp/config/config.go and cmd/thv-operator/api/v1alpha1/ — both json:"fieldName,omitempty" and yaml:"fieldName,omitempty" tags are required on fields that appear in both contexts.
• +kubebuilder:object:generate=true annotation: Required on AuthServerConfig so controller-gen generates DeepCopyInto/DeepCopy. Follow the exact pattern used by IncomingAuthConfig (line 170 of pkg/vmcp/config/config.go).
• Struct definition move (same package): Moving ExternalAuthConfigRef from mcpserver_types.go to mcpexternalauthconfig_types.go is a no-op for all callers because both files are in package v1alpha1. No import changes needed anywhere.
• Validation helper pattern: validateAuthServerConfig() should follow the validateEmbeddingServer() pattern in virtualmcpserver_types.go — a private method on *VirtualMCPServer with a nil-guard at the top, checking the ref name is non-empty when the ref is set.
• SPDX headers: Every Go file must start with // SPDX-FileCopyrightText: Copyright 2025 Stacklok, Inc. and // SPDX-License-Identifier: Apache-2.0. Use task license-fix to add missing headers automatically.

Code Pointers

• cmd/thv-operator/api/v1alpha1/mcpserver_types.go (lines 632–638) — ExternalAuthConfigRef struct definition to be moved (not copied) to mcpexternalauthconfig_types.go. Verify no other struct in mcpserver_types.go references ExternalAuthConfigRef inline before removing.
• cmd/thv-operator/api/v1alpha1/mcpexternalauthconfig_types.go (around line 200, after EmbeddedAuthServerConfig) — destination for ExternalAuthConfigRef. Place it near the other reference-type structs at the bottom of the file.
• cmd/thv-operator/api/v1alpha1/virtualmcpserver_types.go (lines 17–76) — VirtualMCPServerSpec struct. Add AuthServerConfigRef *ExternalAuthConfigRef after EmbeddingServerRef. Add ConditionTypeAuthServerConfigValid constant in the existing const block at lines 211–228.
• cmd/thv-operator/api/v1alpha1/virtualmcpserver_types.go (lines 362–398) — VirtualMCPServer.Validate(). Add r.validateAuthServerConfig() call before return r.validateEmbeddingServer().
• cmd/thv-operator/api/v1alpha1/virtualmcpserver_types.go (lines 400–433) — validateEmbeddingServer() — use as the pattern for the new validateAuthServerConfig() helper.
• pkg/vmcp/config/config.go (lines 89–159) — Config struct. Add AuthServer *AuthServerConfig after Optimizer. Add new AuthServerConfig struct below (near IncomingAuthConfig at line 161).
• pkg/vmcp/config/config.go (lines 186–218) — OIDCConfig struct. Add JwksAllowPrivateIP bool after ProtectedResourceAllowPrivateIP.
• pkg/vmcp/auth/factory/incoming.go (lines 128–159) — newOIDCAuthMiddleware. Update the auth.TokenValidatorConfig literal to map JwksAllowPrivateIP from the OIDC config. Note that AllowPrivateIP is currently set from ProtectedResourceAllowPrivateIP; determine whether JwksAllowPrivateIP should be a separate field on TokenValidatorConfig or reuse the same field (check pkg/auth — the TokenValidatorConfig struct).
• pkg/vmcp/config/zz_generated.deepcopy.go — do not edit manually; regenerate with task gen (from repo root) or the equivalent controller-gen invocation.
• pkg/authserver/config.go — authserver.RunConfig struct (lines 33–74). This is the type wrapped by the new AuthServerConfig. Import path: github.com/stacklok/toolhive/pkg/authserver.

Component Interfaces

New struct in pkg/vmcp/config/config.go:

// AuthServerConfig wraps the auth server's RunConfig for vMCP.
// When non-nil, vMCP starts an embedded OAuth authorization server (Mode B).
// When nil, vMCP uses an external OIDC issuer (Mode A).
// +kubebuilder:object:generate=true
type AuthServerConfig struct {
    RunConfig *authserver.RunConfig `json:"runConfig" yaml:"runConfig"`
}

New field on Config in pkg/vmcp/config/config.go:

// AuthServer configures the embedded OAuth authorization server.
// nil = Mode A (no AS). non-nil = Mode B (AS enabled).
// +optional
AuthServer *AuthServerConfig `json:"authServer,omitempty" yaml:"authServer,omitempty"`

New field on OIDCConfig in pkg/vmcp/config/config.go:

// JwksAllowPrivateIP allows OIDC discovery and JWKS fetches to private IP addresses.
// Required when IncomingAuth.OIDC.Issuer points to an in-cluster service (Mode B loopback).
// Default: false (private IPs rejected, consistent with production security posture).
JwksAllowPrivateIP bool `json:"jwksAllowPrivateIP,omitempty" yaml:"jwksAllowPrivateIP,omitempty"`

New field on VirtualMCPServerSpec in cmd/thv-operator/api/v1alpha1/virtualmcpserver_types.go:

// AuthServerConfigRef references an MCPExternalAuthConfig of type "embeddedAuthServer".
// The referenced resource must exist in the same namespace. nil = Mode A.
// +optional
AuthServerConfigRef *ExternalAuthConfigRef `json:"authServerConfigRef,omitempty"`

New condition type constant in virtualmcpserver_types.go:

// ConditionTypeAuthServerConfigValid indicates whether the auth server config reference is valid
ConditionTypeAuthServerConfigValid = "AuthServerConfigValid"

New validation method on *VirtualMCPServer in virtualmcpserver_types.go:

// validateAuthServerConfig validates the AuthServerConfigRef field.
// Only checks structural validity (name non-empty); cross-resource validation
// (type check, issuer consistency) is deferred to the reconciler in Phase 4.
func (r *VirtualMCPServer) validateAuthServerConfig() error {
    if r.Spec.AuthServerConfigRef != nil && r.Spec.AuthServerConfigRef.Name == "" {
        return fmt.Errorf("spec.authServerConfigRef.name is required when authServerConfigRef is set")
    }
    return nil
}

Updated auth.TokenValidatorConfig construction in pkg/vmcp/auth/factory/incoming.go — confirm the exact field mapping with pkg/auth:

oidcConfig := &auth.TokenValidatorConfig{
    Issuer:            oidcCfg.Issuer,
    ClientID:          oidcCfg.ClientID,
    Audience:          oidcCfg.Audience,
    ResourceURL:       oidcCfg.Resource,
    AllowPrivateIP:    oidcCfg.ProtectedResourceAllowPrivateIP || oidcCfg.JwksAllowPrivateIP,
    InsecureAllowHTTP: oidcCfg.InsecureAllowHTTP,
    Scopes:            oidcCfg.Scopes,
}

Note: If pkg/auth.TokenValidatorConfig has separate fields for resource endpoint and JWKS/discovery private IP allowances, use them separately. If it has a single AllowPrivateIP field (as currently wired), the OR approach above is the correct stopgap until the auth package is extended.

Testing Strategy

Unit Tests

No new test files are required for Phase 1 (all validation unit tests are in Phase 3). However, the following existing tests must continue to pass:

• pkg/vmcp/config/validator_test.go — all existing test cases pass unchanged (Mode A config with nil AuthServer must still be valid)
• cmd/thv-operator/api/v1alpha1/ — any existing virtualmcpserver_types_test.go tests pass unchanged

Structural Validation Tests (Phase 1 — optional but encouraged)

If the team adds early validation coverage, a minimal test for validateAuthServerConfig can go in cmd/thv-operator/api/v1alpha1/virtualmcpserver_types_test.go:

• Nil AuthServerConfigRef — Validate() returns nil (no regression)
• AuthServerConfigRef with non-empty Name — Validate() returns nil
• AuthServerConfigRef with empty Name — Validate() returns error containing "spec.authServerConfigRef.name is required"

Generated Code Checks

• zz_generated.deepcopy.go (in pkg/vmcp/config/) includes DeepCopyInto/DeepCopy for AuthServerConfig
• cmd/thv-operator/api/v1alpha1/zz_generated.deepcopy.go includes updated deepcopy for VirtualMCPServerSpec reflecting the new AuthServerConfigRef field
• CRD YAML in config/crd/bases/ includes authServerConfigRef in the VirtualMCPServer spec schema

Edge Cases

• Config.AuthServer = nil in all YAML serialization round-trips — the field must be absent (omitempty ensures this; verify with a marshaling test or manual inspection)
• VirtualMCPServerSpec.AuthServerConfigRef = nil — no behavior change in reconciler (Phase 4 adds the reconciler logic; Phase 1 must not break the reconciler with a nil pointer)

Out of Scope

• Any runtime behavior change — the vMCP server does not read Config.AuthServer in this phase
• RegisterHandlers method on EmbeddedAuthServer (Phase 2)
• Conditional AS creation in cmd/vmcp/app/commands.go (Phase 2)
• AuthServer *runner.EmbeddedAuthServer field on pkg/vmcp/server.Config (Phase 2)
• Replacing the /.well-known/ catch-all handler (Phase 2)
• validateAuthServerIntegration function and V-01..V-07 rules (Phase 3)
• StrategyTypeUpstreamInject constant and UpstreamInjectConfig struct (Phase 3)
• Operator reconciler cross-resource validation and AuthServerConfigValid condition surfacing (Phase 4)
• CRD-to-config converter changes in converter.go (Phase 4)
• E2E tests (Phase 4)
• Documentation updates to docs/arch/ (Phase 4)

References

• RFC-0053 design document: docs/proposals/THV-0053-vmcp-embedded-authserver.md
• Parent epic: vMCP: add embedded authorization server #4120
• RFC-0052 (multi-upstream IDP, required for full E2E): Auth Server: multi-upstream provider support #3924
• Operator CRD development guide: cmd/thv-operator/CLAUDE.md
• EmbeddedAuthServerConfig (CRD type, already implemented): cmd/thv-operator/api/v1alpha1/mcpexternalauthconfig_types.go (lines 152–200)
• authserver.RunConfig (wrapped by AuthServerConfig): pkg/authserver/config.go (lines 33–74)
• EmbeddedAuthServer (already implemented, used in Phase 2): pkg/authserver/runner/embeddedauthserver.go
• Incoming auth factory (pattern for JwksAllowPrivateIP wiring): pkg/vmcp/auth/factory/incoming.go
• Deepcopy generation guide: Run task gen (repo root) or task operator-generate (operator path)