docs(perf): publish the binary-size and boot-time audit behind #652

[mparrett]

What

Publishes the binary-size and boot-time audit behind #652: a controlled comparison of v1.7.4 with main pinned at ed4ecc2, with six measurement/analysis tools and the raw timing and size data behind the tables.

Size and boot were measured on 2026-07-17 on darwin/arm64 with go1.26.3. The compute comparison was corrected and re-run on 2026-09-17 with go1.26.5: each round now measures both refs and alternates AB/BA order, rather than asking Hyperfine to run two command-sized blocks. The document dates both sets of measurements and keeps the historical endpoint explicit instead of implying that ed4ecc2 is current main.

Why it was measured

A comparison table from another machine reported "+40% startup / +24% binary". Those figures came from a single-run time and an unstripped build, so the audit re-ran both in a controlled setup on one machine. Noisy compute measurements now use paired, order-balanced A/B rounds.

Four results that outlived the table:

The startup regression was real and larger than claimed, and it was already fixed. In-process boot went 744 -> 5499 microseconds by v1.11.0, then 5499 -> 1145 in v1.11.1 once the ir.* pipeline became lazy. Boot tracks the core bundle almost exactly (694 KiB -> 5499 us, 228 KiB -> 1145 us), not the Go binary.

It never appeared on the perf page, and the reason is structural. BenchmarkInitFromLGB entered the ratchet in #355 on 2026-06-29 — one day after the v1.11.0 peak and the same day as the fix. The whole regression window had no boot metric. The dashboard's boot series effectively starts after the spike it would have caught.

The binary headline was an artifact. Stripped, which is what ships, the growth is 10.07 -> 12.64 MiB. The embedded bundle grew only 130 -> 242 KiB, so about 96% of the binary growth is outside the bundle; package attribution points to Go code. Per package: pkg/vm +642 KiB, pkg/rt +435 KiB, and a new 230 KiB go/ast/parser/printer cluster that pkg/rt/gogen.go links into every binary, wasm included, because it carries no build tag.

Compute is slower, with corrected paired estimates. Across 15 paired rounds, alternating AB/BA, ed4ecc2 is +12.8% on fib and +17.6% on tak by paired median versus v1.7.4. The exact round order, timings, and paired ratios are checked in as JSON.

The lever, with its cost

Tagging gogen.go //go:build bootstrap saves 696 KiB (5.4%) stripped and still boots and runs. It also breaks runtime *ir-compile*, whose lazy require reaches gogen transitively. The patch is included as gogen-bootstrap-gate.patch, and section 1 sets out the version that keeps both: decouple ir.lower-go from the pipeline's unconditional :require so only the :go target pulls gogen.

The gaps

The audit misread the ratchet here, and §4 now says so. It reported boot as gated on timing alone; in fact the deterministic allocs/bytes gate landed in #362 on 2026-06-30, two weeks before the audited ref, at a 2% budget tighter than the 5% timing one. The gate's real weakness is #883: machineIndependentBar takes minima across profiles captured at different commits.

Binary and bundle size remain ungated. The stripped binary grew +2.57 MiB and the bundle had already crept 228 -> 242 KiB by ed4ecc2, but nothing in cmd/bench-ratchet measures size today. scripts/nm-package-diff.py is the per-package attribution such a gate would need.

Reproducing

release-compare.sh, boot-e2e.sh, binary-size-walk.sh, and ab-compute.sh build throwaway worktrees and leave the caller's checkout alone. Start with release-compare.sh; only boot-e2e.sh needs Hyperfine. ab-compute.sh uses the checked-in Python paired runner and records every round order and B/A ratio.

[mparrett]

Caveat: testing machine is hot