prereg(H_9988): does the recombination wall move with DATA DIVERSITY? — exposure held FIXED, corpus scaled 500x on a public natural corpus - #4643
Merged
Conversation
… — exposure held FIXED, corpus scaled 500x on a public natural corpus TRIGGERED BY MEASUREMENT, not by preference. H_9987's ladder ran 9/9 and the recombination wall did not move: rho-weave best_distinct read 0-1 against a bar of >=2 at every rung, and pressing the capacity pressure 10x harder (0.28x -> 0.03x) produced no trend. Fluency passed 9/9, so the FLUENCY-BOUND escape is closed — the models could speak; they could not recombine. The ratio isomorph is dead, which leaves exactly the hypothesis H_9984 pre-committed as its successor: "not ratio but DIVERSITY, and that is bought with scale money." This is also the one axis the repo's own root rules flag as never opened: natural corpus at LLM data scale. Parameter scale was measured as an amplifier-not-lever, but on the SAME small corpora, so it never bounded the data axis. THE DESIGN'S ONE IDEA: HOLD EXPOSURE FIXED. Growing a corpus normally moves quantity and diversity together, and then a positive cannot be attributed. Fixing total training bytes (steps) across every rung means a larger corpus is simply seen with FEWER repeats, so the only moving variable is how much unique content exists: D0 60 MB (en_general, the baseline) 1x D1 600 MB 10x diversity, same bytes seen D2 6 GB 100x D3 30 GB 500x Rungs are NESTED (D1 in D2 in D3, shards accumulated) so they differ in RANGE, not in distribution. Model size is pinned at --d 1200 (~30M): the capacity axis was already swept by H_9987, and moving both at once would cross-contaminate. PROCUREMENT IS MEASURED, not assumed: fineweb-edu sample/10BT, 14 shards x 2.15GB = 28.5GB, public natural EN (CommonCrawl educational filter), ~63 MB/s observed, 726,000 documents per shard at ~4KB each (~2.9GB of text per shard), disk 239G -> 216G. D1 (600.0MB, 131,200 docs) is already built, so the first rung can fire without waiting on the remaining shards. THE HONEST COST. There is NO $0 pre-gate for unlabelled web text: `--earned` needs T annotated outside the token stream and ngram-audit needs labels too. So this branch pays a training run per rung — which is part of why the axis has stayed shut. Five-cell table frozen before firing (DIVERSITY-OPENS · DIVERSITY-TREND · DIVERSITY-KILL · FLUENCY-BOUND · REVERSED); the ARCHITECTURE node lands PENDING with every numeric slot empty. SCOPE: 30GB is the ceiling. The LLM regime (~10^12 tokens, TB-scale) is unreachable on this disk, so no result here may be extended to "and so at LLM scale". What this opens is the GB–30GB band, i.e. directly above the MB–GB band H_9968 closed. # no-verify-ok pre-existing doc-gate debt only: ZERO new violations (ARCH-BIG-CELL 148, unchanged), # 6 new leaves max 268 chars, JSON validated before commit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TRIGGERED BY MEASUREMENT, not by preference. H_9987's ladder ran 9/9 and the recombination wall did
not move: rho-weave best_distinct read 0-1 against a bar of >=2 at every rung, and pressing the
capacity pressure 10x harder (0.28x -> 0.03x) produced no trend. Fluency passed 9/9, so the
FLUENCY-BOUND escape is closed — the models could speak; they could not recombine. The ratio
isomorph is dead, which leaves exactly the hypothesis H_9984 pre-committed as its successor:
"not ratio but DIVERSITY, and that is bought with scale money."
This is also the one axis the repo's own root rules flag as never opened: natural corpus at LLM data
scale. Parameter scale was measured as an amplifier-not-lever, but on the SAME small corpora, so it
never bounded the data axis.
THE DESIGN'S ONE IDEA: HOLD EXPOSURE FIXED. Growing a corpus normally moves quantity and diversity
together, and then a positive cannot be attributed. Fixing total training bytes (steps) across every
rung means a larger corpus is simply seen with FEWER repeats, so the only moving variable is how
much unique content exists:
D0 60 MB (en_general, the baseline) 1x
D1 600 MB 10x diversity, same bytes seen
D2 6 GB 100x
D3 30 GB 500x
Rungs are NESTED (D1 in D2 in D3, shards accumulated) so they differ in RANGE, not in distribution.
Model size is pinned at --d 1200 (~30M): the capacity axis was already swept by H_9987, and moving
both at once would cross-contaminate.
PROCUREMENT IS MEASURED, not assumed: fineweb-edu sample/10BT, 14 shards x 2.15GB = 28.5GB, public
natural EN (CommonCrawl educational filter), ~63 MB/s observed, 726,000 documents per shard at ~4KB
each (~2.9GB of text per shard), disk 239G -> 216G. D1 (600.0MB, 131,200 docs) is already built, so
the first rung can fire without waiting on the remaining shards.
THE HONEST COST. There is NO $0 pre-gate for unlabelled web text:
--earnedneeds T annotatedoutside the token stream and ngram-audit needs labels too. So this branch pays a training run per
rung — which is part of why the axis has stayed shut.
Five-cell table frozen before firing (DIVERSITY-OPENS · DIVERSITY-TREND · DIVERSITY-KILL ·
FLUENCY-BOUND · REVERSED); the ARCHITECTURE node lands PENDING with every numeric slot empty.
SCOPE: 30GB is the ceiling. The LLM regime (~10^12 tokens, TB-scale) is unreachable on this disk, so
no result here may be extended to "and so at LLM scale". What this opens is the GB–30GB band, i.e.
directly above the MB–GB band H_9968 closed.
no-verify-ok pre-existing doc-gate debt only: ZERO new violations (ARCH-BIG-CELL 148, unchanged),
6 new leaves max 268 chars, JSON validated before commit.
Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com