Repository navigation
Derive test counts from judge fixtures - #230
Merged
Merged
Conversation
moon-jam
force-pushed
the
fix/validate-judge-case-counts
branch
from
October 3, 2026 13:23
0551500 to
3c652ef
Compare
jserv
reviewed
Oct 3, 2026
moon-jam
force-pushed
the
fix/validate-judge-case-counts
branch
from
October 3, 2026 14:06
3c652ef to
45da8b0
Compare
ColtenOuO
reviewed
Oct 3, 2026
moon-jam
force-pushed
the
fix/validate-judge-case-counts
branch
2 times, most recently
from
October 5, 2026 11:12
d537ebc to
1a7d0b8
Compare
jserv
requested changes
Oct 5, 2026
jserv
left a comment
jserv
left a comment
Contributor
There was a problem hiding this comment.
Rebase latest main branch and resolve conflicts.
Repeated browser totals and literal report comparisons drift when judge cases change. Resolve flow judges before browser startup and reuse their counts. Validate live report totals against current fixtures while comparing passing counts with the saved capture. Keep its total historical so case additions do not require updates.
moon-jam
force-pushed
the
fix/validate-judge-case-counts
branch
from
October 7, 2026 01:47
1a7d0b8 to
896f76c
Compare
Contributor
|
Thank @moon-jam for contributing! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This follows up on the review of #117 by removing hard-coded browser case counts and validating the saved report total against the judge fixtures.
Browser checks previously repeated judge case counts as numeric literals, requiring manual updates whenever cases changed. They now resolve each problem through
web/problem-pages.jsonand read the count from its generated fixture underweb/judges/.This also corrects
tests/golden/report-python.jsonfrom0/4to0/5and checks its total against the Two Sum judge during fixture-only report validation. The offline gate validates that every mapped problem has a nonempty case list. Regression tests cover changing counts, missing problem IDs, malformed case lists, and stale report totals.Tests
node scripts/judge-case-counts.cjsREPORT_PARITY_CHECK_VALIDATE_FIXTURES_ONLY=1 ./scripts/report-parity-check.sh./node_modules/.bin/eslint scripts/browser-check.cjs scripts/judge-case-counts.cjs tests/browser/judge-case-counts.test.jsRefs #117
Summary by cubic
Derives judge case counts from generated fixtures so browser checks and the report parity check no longer drift when cases change.
scripts/judge-case-counts.cjsmaps problem IDs to fixtures and fails when any judge has no cases.Written for commit 896f76c. Summary will update on new commits.