Skip to content

test(perf): no-op probe — measure perf-pr runner noise floor - #13

Closed
mparrett wants to merge 1 commit into
mainfrom
perf/noop-noise-floor
Closed

test(perf): no-op probe — measure perf-pr runner noise floor#13
mparrett wants to merge 1 commit into
mainfrom
perf/noop-noise-floor

Conversation

@mparrett

@mparrett mparrett commented Jul 12, 2026

Copy link
Copy Markdown
Owner

Throwaway PR to measure the same-runner A/B noise floor on paserati's suite, back-porting the let-go investigation (nooga/let-go#445).

The only change is a comment in pkg/vm/object_bench_test.go, so merge-base and head are perf-identical. With the perf label, perf-pr.yml benches base vs head on one runner — every reported delta is pure same-runner measurement noise. Running it a few times shows whether paserati has the same heavy-tailed, anchor-invisible blowout let-go does. Not for merge.

@mparrett
test(perf): no-op probe to measure perf-pr runner noise floor
0124587 
Comment-only change to pkg/vm/object_bench_test.go so the PR's merge-base and
head are perf-identical. Every delta perf-pr reports is same-runner measurement
noise. Back-porting the nooga/let-go#445 investigation to paserati. Throwaway.
@mparrett mparrett added the perf Run the same-runner microbenchmark A/B on this PR label Jul 12, 2026
@github-actions

github-actions Bot commented Jul 12, 2026 •
edited
Loading

Copy link
Copy Markdown

Perf — base vs head, same runner

Base c4c332facd98 vs head 01245871ac29. Anchor-normalized; informational, not a gate.

bench-ratchet — 0 regression(s) > 10.0% budget, 0 missing, 0 new

  • baseline: AMD EPYC 9V74 80-Core Processor / go1.26.0 / linux-amd64
  • current: AMD EPYC 9V74 80-Core Processor / go1.26.0 / linux-amd64
  • anchor: baseline 1.406 ns/op, current 1.406 ns/op (+0.0%)
Benchmark Baseline× Current× Δ% Wall Best since Status
github.com/nooga/paserati/tests.BenchmarkPrototypeMethodAccess/WithDetailedStats/ObjectPrototypeChain 1234451.636 1319467.994 +6.9% 1.86 ms c4c332facd98 ok
github.com/nooga/paserati/tests.BenchmarkPrototypeCacheHitRate 1325384.068 1372009.957 +3.5% 1.93 ms c4c332facd98 ok
github.com/nooga/paserati/tests.BenchmarkPrototypeMethodAccess/WithDetailedStats/ArrayPrototypeMethod 1147696.302 1171782.361 +2.1% 1.65 ms c4c332facd98 ok
github.com/nooga/paserati/tests.BenchmarkPrototypeMethodAccess/WithDetailedStats/StringPrototypeMethod 1114026.316 1128645.804 +1.3% 1.59 ms c4c332facd98 ok
github.com/nooga/paserati/tests.BenchmarkMatrixMult 5538609.531 5573906.828 +0.6% 7.84 ms c4c332facd98 ok
github.com/nooga/paserati/pkg/vm.BenchmarkGetOwn/n=4/first 5.262 5.269 +0.1% 7.41 ns c4c332facd98 ok
github.com/nooga/paserati/pkg/vm.BenchmarkGetOwn/n=8/last 28.250 28.286 +0.1% 39.8 ns c4c332facd98 ok
github.com/nooga/paserati/pkg/vm.BenchmarkGetOwn/n=8/first 5.262 5.268 +0.1% 7.41 ns c4c332facd98 ok
github.com/nooga/paserati/pkg/vm.BenchmarkGetOwn/n=32/round-robin 12.745 12.760 +0.1% 17.9 ns c4c332facd98 ok
github.com/nooga/paserati/pkg/vm.BenchmarkGetOwn/n=1/first 5.267 5.269 +0.0% 7.41 ns c4c332facd98 ok
github.com/nooga/paserati/pkg/vm.BenchmarkGetOwn/n=64/last 11.963 11.956 -0.1% 16.8 ns c4c332facd98 ok
github.com/nooga/paserati/pkg/vm.BenchmarkGetOwn/n=32/first 11.906 11.899 -0.1% 16.7 ns c4c332facd98 ok
github.com/nooga/paserati/pkg/vm.BenchmarkGetOwn/n=64/first 11.771 11.764 -0.1% 16.5 ns c4c332facd98 ok
github.com/nooga/paserati/pkg/vm.BenchmarkGetOwn/n=4/round-robin 10.519 10.512 -0.1% 14.8 ns c4c332facd98 ok
github.com/nooga/paserati/pkg/vm.BenchmarkGetOwn/n=1/round-robin 5.639 5.634 -0.1% 7.92 ns c4c332facd98 ok
github.com/nooga/paserati/pkg/vm.BenchmarkGetOwn/n=16/first 11.764 11.750 -0.1% 16.5 ns c4c332facd98 ok
github.com/nooga/paserati/pkg/vm.BenchmarkGetOwn/n=8/round-robin 17.098 17.077 -0.1% 24 ns c4c332facd98 ok
github.com/nooga/paserati/pkg/vm.BenchmarkGetOwn/n=1/last 5.391 5.381 -0.2% 7.57 ns c4c332facd98 ok
github.com/nooga/paserati/tests.BenchmarkFibPlaceholderRun 731872158.606 729863064.723 -0.3% 1.03 s c4c332facd98 ok
github.com/nooga/paserati/pkg/vm.BenchmarkGetOwn/n=32/last 12.134 12.098 -0.3% 17 ns c4c332facd98 ok
github.com/nooga/paserati/pkg/vm.BenchmarkGetOwn/n=16/last 12.006 11.956 -0.4% 16.8 ns c4c332facd98 ok
github.com/nooga/paserati/tests.BenchmarkPrototypeMethodAccess/WithPrototypeCache/ObjectPrototypeChain 1183623.755 1174010.669 -0.8% 1.65 ms c4c332facd98 ok
github.com/nooga/paserati/pkg/vm.BenchmarkGetOwn/n=16/round-robin 12.831 12.717 -0.9% 17.9 ns c4c332facd98 ok
github.com/nooga/paserati/pkg/vm.BenchmarkGetOwn/n=64/round-robin 13.051 12.923 -1.0% 18.2 ns c4c332facd98 ok
github.com/nooga/paserati/pkg/vm.BenchmarkGetOwn/n=4/last 15.398 15.213 -1.2% 21.4 ns c4c332facd98 ok
github.com/nooga/paserati/tests.BenchmarkPrototypeMethodAccess/WithPrototypeCache/ArrayPrototypeMethod 1256091.038 1232317.212 -1.9% 1.73 ms c4c332facd98 ok
github.com/nooga/paserati/tests.BenchmarkPrototypeMethodAccess/Baseline/StringPrototypeMethod 1210399.004 1184798.009 -2.1% 1.67 ms c4c332facd98 ok
github.com/nooga/paserati/tests.BenchmarkPrototypeMethodAccess/Baseline/ArrayPrototypeMethod 1417272.404 1338169.275 -5.6% 1.88 ms c4c332facd98 ok
github.com/nooga/paserati/tests.BenchmarkPrototypeMethodAccess/Baseline/ObjectPrototypeChain 1644120.910 1504844.239 -8.5% 2.12 ms c4c332facd98 ok
github.com/nooga/paserati/tests.BenchmarkPrototypeMethodAccess/WithPrototypeCache/StringPrototypeMethod 1401487.909 1269135.135 -9.4% 1.78 ms c4c332facd98 ok

Run details →

@mparrett

mparrett commented Jul 12, 2026

Copy link
Copy Markdown
Owner Author

Closing — noise-floor probe that motivated the median-of-N gate; its data lives in nooga#21 and the gate is on PR #14. Scaffolding, no longer needed.

@mparrett mparrett closed this Jul 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

perf Run the same-runner microbenchmark A/B on this PR

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@mparrett