Skip to content

Commit 15b23dd

Browse files
norandomclaude
andcommitted
docs(spec): final validation of the agnostic driver (task 8.3, R80)
Validation report, design Amendment 3, research entry and index, the single handover and the task-group handover. Every task of the spec is ticked; Requirement 5.7 is recorded as met in part (no signal handler). The validation was done inline by the implementer. Co-Authored-By: Claude <noreply@anthropic.com>
1 parent c2a0a1b commit 15b23dd

8 files changed

Lines changed: 300 additions & 43 deletions

File tree

‎.kiro/handover.md‎

Lines changed: 38 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -1,43 +1,45 @@
1-
# Handover: driver built through task group 6 and run on the gate scopes; Slice B measured on two models, 2026-11-10
1+
# Handover: driver complete through validation, old adapters deleted; Slice B measured with two optimisers, 2026-11-11
22

3-
Updated 2026-11-10, about 23:30 Europe/Berlin, at the end of an unattended evening session. This file is the
4-
single handover and action list. The current sections below govern continuation; dated history follows for
5-
evidence and earlier interpretations, not as an active implementation queue.
3+
Updated 2026-11-11, about 08:35 Europe/Berlin. This file is the single handover and action list. The current
4+
sections below govern continuation; dated history follows for evidence and earlier interpretations, not as an
5+
active implementation queue.
66

77
## Current state
88

9-
Start with the [task 32 handover](task_32_handover.md) and the
10-
[overnight plan](overnight-plan-2026-11-10.md), which lists the open tasks and the owner's decisions.
11-
12-
1. **Top priority: the agnostic driver** (spec
13-
[agnostic-annotation-driver](specs/agnostic-annotation-driver/tasks.md); requirements, design and tasks
14-
approved 2026-11-10). Task groups 1 to 6 and tasks 8.1 and 8.2 are done; main is green on five platforms.
15-
- `lesscode-core deconvolute analyze | mutate | run | admit-pack` work through the driver for Rust,
16-
Python, TypeScript and Java ([docs/driver.md](../docs/driver.md)).
17-
- The engine reproduces the prototype's sites and mutants exactly
18-
([R67](../docs/research/2026-11-10/13-native-driver-engine-parity.md)), and `deconvolute run` gives
19-
every one of 556 mutants on the five gate scopes the class the prototype gave
20-
([R73](../docs/research/2026-11-10/19-native-driver-batches-on-gate-scopes.md)). No source tree was
21-
changed by any run.
22-
- Not done, and waiting for the owner: task group 7 (delete the six old adapter files, restate the
23-
specimen results, README and adoption guide) and task 8.3 (final validation). No release contains the
24-
driver.
25-
2. **Second priority: Slice B,** hardening against LLM inference drift (`constraint-guided-compression`).
26-
Started and measured this evening; entries R66 and R68 to R74.
27-
- Baseline on `claude-opus-5-5`: every pack admitted at first, but Rust packs differed between runs
28-
unnoticed (R68); with hand-written Rust controls no first Rust reply is admitted (R69).
29-
- Optimising the guide with a GEPA engine copied from `dsrust` (tool `tools/guide-optimizer/`, R70): on
30-
Opus every first reply is then admitted for all four languages and three languages match the shipped
31-
packs; Rust still differs from the hand pack in four fixed points, so the guide is not adopted (R71).
32-
- `mistral-large-4`: 5 of 20 first replies admitted, mostly failing on node kinds that do not exist
33-
(R72); optimisation finds no improvement (R74).
34-
- Open for the owner: the four Rust points of R71; whether to give the author the grammar's node kinds;
35-
a `MIPROv2` comparison; the annotation layer, which is untouched.
36-
- Model spend recorded at list prices: about 15.2 of 20 USD.
37-
3. **Rules that held this session:** no deletion of the old adapters, no release, no change to what the
38-
shipped packs make the driver do except the two Python operator pairs the owner approved, no management
39-
of the Podman machine, containers removed after each batch, keys never printed or committed (`.env` is
40-
git-ignored; the Mistral key was pasted into the chat and is in that transcript).4. **Release:** `v1.0.2` is published (2026-11-10) as the first signed
9+
Start with the [task 33 handover](task_33_handover.md).
10+
11+
1. **The agnostic driver** (spec [agnostic-annotation-driver](specs/agnostic-annotation-driver/tasks.md)):
12+
every task is done and validated
13+
([report](specs/agnostic-annotation-driver/validation-report.md),
14+
[R80](../docs/research/2026-11-11/06-agnostic-driver-final-validation.md)).
15+
- `lesscode-core deconvolute analyze | mutate | run | admit-pack` for Rust, Python, TypeScript and Java
16+
([docs/driver.md](../docs/driver.md)). The six text-matching adapter files are deleted.
17+
- The specimen gates' "4 of 4 killed" figures are restated
18+
([R75](../docs/research/2026-11-11/01-specimen-controls-rerun-through-driver.md)): for Rust they were
19+
written into the gate script; with the projects' own tests four of the five reproducible controls are
20+
killed and one survives. R49 to R53 carry dated amendments.
21+
- Met in part: Requirement 5.7 (no signal handler). No release contains the driver.
22+
- Lesson of the day: copies must not share a build directory. On this machine `CARGO_TARGET_DIR` is set
23+
globally; a Rust run needs `"env": {"CARGO_TARGET_DIR": "target"}` in its configuration. The same
24+
shared directory can hand this repository's own tests a stale library after experiments in a copy;
25+
`cargo clean -p lesscode-core` clears it.
26+
2. **Slice B,** hardening against LLM inference drift (`constraint-guided-compression`): entries R66 and R68
27+
to R79.
28+
- Owner decisions of 2026-11-11: no Rust point of R71 goes into the shipped pack; node kind list tried on
29+
both models; stop for model spend raised to 40 USD; MIPROv2 comparison run.
30+
- Node kinds as input (R76, R78): invalid node kinds fall from 10 to 1 of 20 first replies on
31+
`mistral-large-4`, admission stays at 5 and 6 of 20; no change on `claude-opus-5-5`. Not adopted.
32+
- MIPROv2-style search on Opus (R77, R79): Rust admitted 0 of 5 in the final judgement, where GEPA's
33+
guide got 5 of 5 (R71). Nothing adopted.
34+
- Measured so far: drift between runs, between models, under two optimisers. Not measured: the annotation
35+
layer; one model over time. Next step is the owner's choice between an annotation-layer baseline and
36+
closing the pack-authoring question.
37+
- Model spend recorded at list prices: about 25.5 of 40 USD.
38+
3. **Open for the owner:** a release with the driver; a signal handler; `PureCoreVerifier::compare` on the
39+
adapters and `NativeMutantGenerator::generate`, which report success without running anything; operators
40+
for a changed argument and a changed literal in a call; a CI job for `tools/guide-optimizer/`; rotating the
41+
Mistral key, which was pasted into a chat.
42+
4. **Release:** `v1.0.2` is published (2026-11-10) as the first signed
4143
release: archives carry zipsign signatures, `SHA256SUMS.sig` and a
4244
provenance attestation, all verified after publication. It contains the
4345
fail-loudly fixes (item 5). The owner holds the signing key and approves

‎.kiro/specs/agnostic-annotation-driver/design.md‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -388,6 +388,24 @@ list of shipped packs so that no other driver file names a language. Packs carry
388388
`store.rs` 59. The driver is about 2,900 production lines in eighteen files against an estimate of about
389389
2,400; every file is within the per-file budget.
390390

391+
### Amendment 3, 2026-11-11 (task group 7): the migration as carried out
392+
393+
- The adapter traits are answered by one file, `adapters/driver/contract.rs`, which the three adapter modules
394+
call; the design had each adapter implement them. No Python adapter struct was added (Amendment 2).
395+
- `MutantOutcome` has a new variant, `NotRun { mutants }`. Generation through the trait runs no test, so a
396+
control with a driver mutant is reported as not run. The removed operators reported "killed" when the text
397+
had changed; that meaning is gone on purpose.
398+
- The mapping of step 3 is `contract::operators_of`. `SilentNone` maps to a dropped statement or a swapped
399+
returned literal, `MissingFailureState` to the removal or rewrite of a failure signal. Seven of the twelve
400+
hand-made controls of the specimen gates have no counterpart and are recorded as lost (R75).
401+
- Step 4 could not be done as written for Rust: the candidate specimen is not part of any build. The rerun
402+
uses the original code in `native/src/recording.rs`. For Java the rerun used the JDK and Maven on the
403+
machine instead of a container, because the container engine was not running.
404+
- Not touched, and reported to the owner: `PureCoreVerifier::compare` on the adapters and
405+
`NativeMutantGenerator::generate` report success without running anything. They are outside the six files
406+
this spec deletes.
407+
- The driver is now twenty files and about 3,660 lines including tests in the files; the largest is 348.
408+
391409
## Contradictions and disposition
392410

393411
| # | Contradiction | Source authority | Disposition |

‎.kiro/specs/agnostic-annotation-driver/spec.json‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
{
22
"feature_name": "agnostic-annotation-driver",
33
"created_at": "2026-11-10T10:30:00+01:00",
4-
"updated_at": "2026-11-10T19:00:00+01:00",
4+
"updated_at": "2026-11-11T08:30:00+01:00",
55
"language": "en",
6-
"phase": "tasks-generated",
6+
"phase": "implementation-complete",
77
"approvals": {
88
"requirements": {
99
"generated": true,
@@ -37,7 +37,11 @@
3737
"docs/research/2026-11-10/08-annotation-driver-parallel-batches.md",
3838
"docs/research/2026-11-10/09-annotation-driver-attempt2-setup.md",
3939
"docs/research/2026-11-10/10-annotation-driver-gate-attempt2.md",
40-
"docs/research/2026-11-10/11-annotation-driver-rust-sizing-spike.md"
40+
"docs/research/2026-11-10/11-annotation-driver-rust-sizing-spike.md",
41+
"docs/research/2026-11-10/13-native-driver-engine-parity.md",
42+
"docs/research/2026-11-10/19-native-driver-batches-on-gate-scopes.md",
43+
"docs/research/2026-11-11/01-specimen-controls-rerun-through-driver.md",
44+
"docs/research/2026-11-11/06-agnostic-driver-final-validation.md"
4145
],
42-
"notes": "2026-11-10: owner set priorities (agnostic driver first, inference-drift hardening second) and approved a new spec with mutation as the first annotation consumer. Brief and prototype gate written for review; gate not run. No requirements, design or tasks generated. Owner go (proceed) 2026-11-10: gate checklist step 1 scope screening done (R57, gate Amendment 1: four scopes admitted, control expectation corrected). Arm D (rules as data, model as offline rule author) approved by the owner 2026-11-10. Owner requirement 2026-11-10: hard-coded operators to be removed completely (gate Amendment 2b). Step 2 done (R58): validator controls. Gate attempt 1 completed 2026-11-10 with outcome revise (R61): Rust, Python and Java pass in at least one arm, TypeScript fails in every arm; model-written packs admitted and equivalent to hand-written ones. 21 model calls. Attempt 2 ran 2026-11-10 after the owner chose the full revision (R63, R64): arm D (rules as data, model-written packs for held-out languages) meets every mechanism row on four languages and five scopes; arm A misses yield in one TypeScript run; apparatus ceiling is the open row and an owner decision. Owner decision 2026-11-10: port to Rust, single runtime, no second runtime; breaking codebases must be avoided (copies only, verified restore). Feasibility report: docs/reports/agnostic-driver-feasibility.md. Owner approved 2026-11-10 the port scope (mutation and analysis together), a sizing spike as the first design step, and the cap decision after the spike. Requirements generated and approved by the owner 2026-11-10 (basis: \"make sure we don't forget slice B this time. other than that approved\"); criterion 9.6 added to keep Slice B scheduled. Sizing spike done (R65): exact parity, core 898 canonical lines vs 753 removable. Owner decision 2026-11-10: the driver stays outside the 7,173-line cap's counted set and is governed by per-file budgets. Design in progress (batch runner next)."
46+
"notes": "2026-11-10: owner set priorities (agnostic driver first, inference-drift hardening second) and approved a new spec with mutation as the first annotation consumer. Brief and prototype gate written for review; gate not run. No requirements, design or tasks generated. Owner go (proceed) 2026-11-10: gate checklist step 1 scope screening done (R57, gate Amendment 1: four scopes admitted, control expectation corrected). Arm D (rules as data, model as offline rule author) approved by the owner 2026-11-10. Owner requirement 2026-11-10: hard-coded operators to be removed completely (gate Amendment 2b). Step 2 done (R58): validator controls. Gate attempt 1 completed 2026-11-10 with outcome revise (R61): Rust, Python and Java pass in at least one arm, TypeScript fails in every arm; model-written packs admitted and equivalent to hand-written ones. 21 model calls. Attempt 2 ran 2026-11-10 after the owner chose the full revision (R63, R64): arm D (rules as data, model-written packs for held-out languages) meets every mechanism row on four languages and five scopes; arm A misses yield in one TypeScript run; apparatus ceiling is the open row and an owner decision. Owner decision 2026-11-10: port to Rust, single runtime, no second runtime; breaking codebases must be avoided (copies only, verified restore). Feasibility report: docs/reports/agnostic-driver-feasibility.md. Owner approved 2026-11-10 the port scope (mutation and analysis together), a sizing spike as the first design step, and the cap decision after the spike. Requirements generated and approved by the owner 2026-11-10 (basis: \"make sure we don't forget slice B this time. other than that approved\"); criterion 9.6 added to keep Slice B scheduled. Sizing spike done (R65): exact parity, core 898 canonical lines vs 753 removable. Owner decision 2026-11-10: the driver stays outside the 7,173-line cap's counted set and is governed by per-file budgets. Design and tasks approved by the owner 2026-11-10. Task groups 1 to 6, 8.1 and 8.2 implemented 2026-11-10 (R67, R73). Owner decision 2026-11-11: do all of task group 7. Six adapter files deleted, specimen results restated (R75), documentation updated, final validation done inline by the implementer (R80, validation-report.md): complete, with Requirement 5.7 met in part (no signal handler). No release contains the driver."
4347
}

‎.kiro/specs/agnostic-annotation-driver/tasks.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -170,7 +170,7 @@ Rules for every task group (owner rules of 2026-11-07 and 2026-11-10):
170170
the narrowed statement that only `deconvolute run` executes a project's commands, in copies.
171171
- _Requirements: 7.5, 8.6_
172172

173-
- [ ] 8. Validation and close
173+
- [x] 8. Validation and close
174174
- [x] 8.1 Run one full batch per language with the native driver
175175
- The five gate scopes through `deconvolute run`; classes compared with the attempt 2 receipts; every
176176
difference explained. Linux-only runs use containers; the Podman machine is not managed by the agent.
@@ -179,7 +179,7 @@ Rules for every task group (owner rules of 2026-11-07 and 2026-11-10):
179179
- The existing CI matrix through the Dagger configuration is green with the driver; binary size and build
180180
time recorded against R65.
181181
- _Requirements: 8.5, 8.6_
182-
- [ ] 8.3 Final validation and Slice B check
182+
- [x] 8.3 Final validation and Slice B check
183183
- `$kiro-validate-impl`; a research entry with the outcome; confirm that the Slice B baseline has a dated
184184
status (run, scheduled with a date, or stopped by the owner). The spec is not reported complete without
185185
it.

0 commit comments

Comments
 (0)