fix(sdd): rule and continue — non-catastrophic conflicts get ledgered rulings, not blocking questions - #2077
Conversation
… rulings, not blocking questions A donated session sat dormant 8h48m waiting for a plan-conflict answer that cost ~zero tokens to decide. Wrong-ruling rework is bounded; stalls are not. This encodes the never-stall doctrine: plan conflicts, ambiguities, and cap exceptions get a controller ruling recorded in the ledger and work proceeds; only irreversible/destructive actions, security-sensitive actions, out-of-worktree side effects (merge/push/ publish), and totally-broken plans remain hard stops. Rulings surface in the Finish report instead of as mid-run questions. Evals: 3/3 no-stall vs control 3/3 stall-at-preflight on a seeded-conflict SDD plan; catastrophic guard 5/5 (every rep reaching a seeded DROP TABLE step refused it); re-validated 3/3 after rebase onto the current fix-PR text; composes cleanly with the evidence-bearing preflight treatment. Claude-Session: https://claude.ai/code/session_0185AJr98gHx5EmwqNeft4Sy
|
@arittr Jesse asked me to request your eval on this one. What to eval it for:
Full pre-registration, per-rep grades, and the adjudication pass are public: https://github.com/prime-radiant-inc/superpowers-autoresearch ( — Claude (Claude Code + Claude Fable 5, on Jesse's behalf at his direction) |
The breaker's two ledger formats wrote lowercase 'ruling' (parked findings, load-bearing adjudications), so the Finish section's collect-every-`Ruling:`-line step missed exactly the rulings made under the most pressure. Field evidence from an independent eval rep: a breaker-cap run adjudicated correctly, wrote everything to the plan-scoped ledger, deleted the workspace at finish, and left no durable trace of the adjudication. Capitalize the two breaker formats to the canonical token, and make the finish roll-up explicitly exhaustive across preflight, parked, and breaker rulings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Independent validation from my environment, as requested — posting post-merge for the record. Setup: quorum A/B (3 reps × 7 scenarios, codex lane, dev vs the full 7-PR stack, both arms on the same binary — host codex 0.144.4, so V2-specific claims were validated separately in live probes on codex-cli 0.146.0 with subscription auth), plus a fresh 2-task SDD run on 0.146. Full numbers below for this PR.
Validation performed by Claude Fable 5 (Claude Code) at my direction. |
…ules on what it surfaces The pre-Task-1 conflict scan currently permits 'the scan is clean' with no evidence the scan happened — mined sessions show controllers skipping straight to dispatch and plan conflicts surfacing mid-execution as blocking questions. Requiring the scan to emit one row per task pair sharing a file/interface and one row per task's self-consistency turns the claim into an artifact; in controlled evals the table appeared 3/3 with conflicts surfaced pre-dispatch, and the mechanism held 3/3 when composed with the never-stall ruling change (obra#2077). Claude-Session: https://claude.ai/code/session_0185AJr98gHx5EmwqNeft4Sy
Who is submitting this PR? (required)
claude-fable-5)What problem are you trying to solve?
Autonomous subagent-driven-development runs stall on questions a controller could decide. The motivating session (donated, in our eval corpus): an SDD run sat dormant 8h48m waiting for a plan-conflict answer whose resolution cost ~zero tokens — the stall burned a working day of wall-clock and human attention. Our mined-session corpus shows the same shape repeatedly: the current SKILL.md text routes plan conflicts, contract ambiguities, and cap exceptions to "ask your human partner," which in an unattended run means the session parks until a human notices. Wrong-ruling rework is bounded and cheap; stalls are not.
In controlled evals, unpatched controllers stalled at the pre-flight batched question in 3/3 reps on a seeded-conflict plan (and 15/15 in the wider baseline set), completing only Task 1 before parking the rest of the plan indefinitely.
What does this PR change?
skills/subagent-driven-development/SKILL.md(+42/−19, one file): non-catastrophic conflicts get a controller ruling recorded in the ledger and work proceeds; a "Rulings I made" section in the finish report surfaces every ruling for human review at the end. Four classes remain hard stops: irreversible/destructive actions, security-sensitive actions, out-of-worktree side effects (merge/push/publish), and plan defects that make all forward progress speculative.Is this change appropriate for the core library?
Yes — it modifies core SDD controller behavior that applies to any project using subagent-driven-development, regardless of domain. No new dependencies, no tool- or project-specific content.
What alternatives did you consider?
Does this PR contain multiple unrelated changes?
No — one file, one behavioral change and its supporting text (process digraph relabels, fix-loop escalation path, finish-report surface all encode the same rule).
Existing PRs
devindependent of them; merge order is free). No open or closed PR addresses controller stalls on plan conflicts.Environment tested
New harness support (required if this PR adds a new harness)
N/A — no new harness.
Evaluation
DROP TABLE ... CASCADEagainst a shared staging database) that must STILL stop.DROP TABLEstep refused/blocked it; zero destructive commands issued in any rep. Re-validated 3/3 after rebasing onto the current fix-PR text (the re-validation battery is what gates this submission), and the composition with the evidence-bearing preflight treatment passed both mechanisms 3/3.logs/2026-08-01-queue-campaign.md(Task 11 pre-registration, grades, adjudication) andreports/2026-08-queue-campaign.md§2.https://claude.ai/code/session_0185AJr98gHx5EmwqNeft4Sy