feat: Thompson Sampling learning router (addresses #445)

[johnkarlhill]

Summary

This proposes a Thompson Sampling bandit as a new routing algorithm that learns from outcomes — directly addressing #445 ("conversation routing without a verifier").

Unlike the existing algorithms (static thresholds, LLM-as-judge, random split), this one improves as traffic flows: per-model posterior distributions over quality are sampled at routing time (exploration vs. exploitation trade-off) and updated from observed outcomes.

Design Sketch

The implementation composes entirely within Switchyard's existing trait model — no new traits required:

ThompsonSamplingAlgorithm (new Algorithm constructor)
  ├── ThompsonSamplingClassifier (Classifier<S> — samples target from Beta posteriors)
  ├── BanditFeedbackProcessor (Processor<S> — updates posteriors from ModelResponse events)
  └── targets (model list)

Feedback signal: V1 uses tool-result signals (binary success/failure from Switchyard's existing tool signals) via an OutcomeScorer trait with a ToolSignalOutcomeScorer default. A richer scorer (e.g., quality-gate-based) can upgrade the signal later without touching the bandit.

Cold-start: config-driven prior per target (default alpha=1, beta=1 = uniform).

Scope cuts (v1, documented as future work):

• No persistence across restarts (Switchyard has no storage layer yet)
• No task-type conditioning (key on ModelId only)
• No session affinity (per-turn sampling)

Dependency: would introduce rand_distr for Beta sampling (idiomatic Rust, more correct than hand-rolled gamma/Beta).

Open Questions

Before writing code, I'd like maintainer input on:

1. Crate placement: crates/libsy (alongside existing algorithms) vs. a new switchyard-bandit crate? I lean toward libsy per the crate-placement rules.

2. Sequencing: is a tool-signal feedback default acceptable, or would maintainers prefer a quality-gate scorer to land first (ci: gate nemo-switchyard release publishing on tags #2 before fix: fix repo links #1) for richer feedback signals?

3. rand_distr dependency: acceptable for Beta sampling, or should sampling be built on rand alone (e.g., gamma sampling via rand) to avoid the new dep?

4. Prior semantics: is a config-driven prior per target the right cold-start model, or should priors derive from something Switchyard already tracks?

Background

I maintain Threshold — a TypeScript LLM routing monorepo with a Thompson Sampling bandit, quality gate, vibe detection, and quantization-aware scoring. Switchyard's backing and community make it the right home for these innovations. This is the first of four planned contributions (see contribution plan).

Happy to align on scope, naming, and structure before opening a PR.

[panpan0000]

Thanks for the proposal! But I'm not sure it fully addresses #445.

The core problem in #445 is that conversation tasks have no verifiable outcome — there's no specific rule card, no tool call, no test result, no pass/fail signal. Your v1 design uses tool-result signals as feedback, which works great for agentic/coding tasks, but conversation tasks don't produce those signals at all. So the bandit wouldn't have anything meaningful to learn from.

[johnkarlhill]

Thanks for the sharp feedback, you're right.

The v1 tool-signal scorer only works for agentic tasks with verifiable outcomes. Conversation tasks (chat, Q&A, writing, tutoring) don't produce those signals, so the bandit would be learning from nothing on exactly the use case #445 is about.

Revised plan: QualityGateProcessor first, then ThompsonSampling.

The quality gate evaluates response quality directly: uncertainty, missing references, contradictions, genericity. No tool calls or test results needed. Works for both agentic and conversation tasks. Landing it first gives the bandit a feedback signal that actually covers conversation routing.

If you're open to this approach, I'll draft the QualityGateProcessor spec and open a PR. The 4 checks are response-level, not tool-level. Let me know how you wish to proceed.