FEAT Add multi-label true/false scoring with explicit label selection

[biefan (biefan)]

Description

Related to #2565.

A classifier can answer several independent questions in one inference, but a single MessageTrueFalseScorer aggregates them into one boolean. WildGuard currently exposes one selected verdict and keeps the other judgments in metadata. Users cannot query or evaluate those secondary judgments as normal scores without making separate classifier calls.

This adds an opt-in multi-label contract and a working WildGuard implementation. For example, one successful classification of a harmful request that receives a refusal can now produce:

score_category	Verdict	Persisted independently
harmful_request	True	Yes
response_refusal	True	Yes
harmful_response	False	Yes

The three scores have distinct IDs and reference the same retained classifier observation. Choosing harmful_response as the objective therefore yields False, regardless of the other labels' True values.

API and design choices

• Separate result family. MultiLabelTrueFalseScorer declares the output labels and validates exactly one category per verdict. MessageMultiLabelTrueFalseScorer adds the existing message pipeline and aggregates supported pieces within each label. Unknown, duplicate, or missing labels are rejected before persistence. Non-applicable evidence still returns [].
• Existing score representation. Labels use Score.score_category=[label], so scores retain their existing IDs, evidence anchors, expectations, serialization, and observation links. There is no schema migration. The existing category query is fixed to match complete JSON-array elements, case-insensitively; comparing the entire array with a string previously missed these stored scores.
• Explicit single-label projection. TrueFalseScoreSelector adapts a named label to TrueFalseScorer for attacks, boolean wrappers, and objective evaluation. Its identity includes the source configuration and selected label, and its batch path retains the source target's rate-limit policy. Raw multi-label scorers cannot silently enter single-verdict consumers or float thresholding.
• WildGuard implementation. WildGuardMultiLabelScorer uses the shared transport, three-label parser, prompt template and context resolution. It makes one classifier call per supported text piece, excluding retries for malformed responses. Any label may return N/A without discarding the other judgments or triggering a retry; it becomes UNDETERMINED. Fully blocked/unreadable evidence also remains undetermined for every label.

classifier = WildGuardMultiLabelScorer(chat_target=wildguard_target)
scores = await classifier.score_async(scorable=evidence)  # Saves all three labels.

objective_scorer = TrueFalseScoreSelector(
    scorer=classifier,
    label="harmful_response",
)

Persistence and compatibility: a selector follows the existing wrapper contract and persists only its selected projection. Call the multi-label root directly to save every label. Separate selector calls do not share cached inference. Existing WildGuardScorer(label=...) and single-verdict scorers retain their APIs and verdict behavior; only WildGuard's unchanged context-resolution logic is shared with the new implementation.

Tests and Documentation

• 45 new cases cover independent persistence/querying, JSON round trips, per-label AND/OR aggregation, invalid output contracts, abstentions, unavailable evidence, shared observations, loose-content anchors, batch context isolation, explicit selection, wrapper composition, rate-limit enforcement, and label-specific evaluation. An attack-level test confirms the selected label controls the actual attack outcome rather than the first classifier result.
• WildGuard tests exercise the real completion target, normalizer, parser and SQLite memory, mocking only the external SDK response. They verify one successful response produces three scores and one shared observation, and malformed-response retries do not duplicate scores.
• Category-query regressions fail against unmodified upstream and pass here. SQLite behavior and SQL Server query construction/bound parameters are covered; no live SQL Server was used.
• Focused scorer, memory, registry and lazy-import suite: 589 passed.
• make unit-test (Python 3.11, default dependencies): 20,464 passed, 146 skipped, 1 failed. The sole failure is the pre-existing test_get_seed_dataset_summaries_follows_a_trailing_blank_insensitive_collation in tests/unit/memory/memory_interface/test_interface_seed_prompts.py, independently reproduced at unmodified base f65263e using the same environment.
• All applicable pre-commit hooks passed with all optional dependencies installed, including repository-wide type checking and documentation validation.

The new offline notebook was executed with the checkout's virtual-environment kernel. Its retained output shows 1 classifier call, 3 persisted scores, 1 shared observation, and no additional call when reading saved categories. The paired Python source matches the notebook. The framework documentation and scoring navigation are updated.

uv run pytest tests/unit/score/test_multi_label_true_false_scorer.py tests/unit/score/test_wildguard_multi_label_scorer.py -q
uv run jupytext --to ipynb --execute --set-kernel python3 doc/code/scoring/6_multi_label_true_false_scorers.py

Validation is local and offline; no live WildGuard endpoint, model weights or paid model calls were used.