RFC: Horizontal Scaling for vMCP and Proxy Runner

[jerm-dro]

THV-0047: Manual Horizontal Scaling for vMCP and Proxy Runner

This RFC defines an approach to enable safe horizontal scale-out of vmcp and thv-proxyrunner by externalizing session state to a shared Redis store and implementing session-aware routing at each layer.

Key Sections

• Background (§1): Deployment architecture, session management infrastructure (transport sessions, vMCP MultiSession, auth server Redis pattern), current client-IP affinity workaround, and the inherent constraint of stateful backends.
• Problems (§2): Session-to-pod affinity issues at both vMCP and proxyrunner layers, and current scaling limitations (hardcoded replica counts, no CRD fields).
• Scope (§3): CRD replica fields, Redis session storage, horizontal scale-out/in for vMCP and proxyrunner (SSE + streamable-http), graceful shutdown. Out of scope: stdio scaling, smart routing at vMCP, auto-scaling policy, backend StatefulSet scale-in.
• High-Level Solution (§4): Redis session records (vMCP and proxyrunner), request routing logic at each layer, Redis configuration via CRDs.
• Requirements (§5): Success criteria organized by component (vMCP, proxyrunner, operator, deployment, security).
• Required Changes (§6): 16 concrete code changes mapped to requirements, organized by component:
  • CRD/Operator (RC-1 to RC-5): Replica fields, session storage config, reconciler updates, Redis injection
  • Transport Session Layer (RC-6, RC-7): Redis Storage backend, storage selection wiring
  • vMCP (RC-8 to RC-10, RC-16): Metadata persistence, session reconstruction via RestoreSession, LRU eviction, backend expiry sync
  • Proxyrunner (RC-11 to RC-13): Configurable StatefulSet replicas, session-aware pod routing, LRU eviction
  • Operational (RC-14): Graceful shutdown (SIGTERM handling, terminationGracePeriodSeconds)
  • Security (RC-15): Cross-pod hijack prevention via persisted token binding

Related

• toolhive#3986 - Sticky sessions on operator-created Services
• toolhive#3992 - ClusterIP service with SessionAffinity for backend
• THV-0038 - Session-Scoped Client Lifecycle for vMCP
• toolhive#1589 - Scaling stdio within k8s

[ChrisJBurns]

Looks ok, couple of comments, maybe some comments that are blockers. The HMAC stuff and subject-only comments I can relay to others expertise if they are blocking or non-issues. I'm more interested in how the ProxyRunner scales at all without vMCP - or if we even want to make a more controversial decision to mandate vMCP for use at scale?

Also, should we defined what observability looks like for this? Redis ops, routing decisions/cross-pod proxy success, distributed trace prop across pod boundaries etc? Or is that a later thing? (this is fine too)

[eleftherias]

This makes sense to me, I have no further comments.
note: I don't have the context to review section 6. Required Changes, but the rest looks good

[yrobla]

left some non blocker comments, but approving

[JAORMX]

excellent initiative