test(ir): Pin transform tests with structural goldens, not IR substrings - #2260
test(ir): Pin transform tests with structural goldens, not IR substrings#2260Hzfengsy wants to merge 1 commit into
Conversation
Twelve tests across three transform test files used a printed-IR substring
as their only oracle, leaving everything outside the grepped tokens
unchecked. Replace them with whole-program comparisons.
test_auto_tile_matmul_l0.py: the three `..._not_folded` guards asserted the
identical pair `"pl.tile.cast(" in printed` / `"pl.tile.assemble(" not in
printed`, so all three passed on all three programs and could not
distinguish the rejection reason each was written to guard. They are pure
no-ops, so add `_assert_unchanged_by_pass` (mirroring the helper in
test_optimize_orch_tensors.py) and pin the whole program. The golden is a
fresh prerequisite-only lowering of `Before`, so the pass under test never
runs on the right-hand side.
test_optimize_orch_tensors.py: convert the five remaining substring tests to
hand-written `Expected` goldens normalized by `_run_prereqs_only`. One of
them, test_aggregate_output_preserves_existing_pure_input_window, already
carried a complete 67-line `Expected` that was never referenced; wire it up
and fix the two defects that had stalled it (locals spelled with `__`,
which auto-naming rejects during prerequisite normalization, and a slice
order that did not match the pass's data/header/out emission order).
test_paged_gather.py: had no structural comparison at all. Add a
`_build_expected` golden builder mirroring `_build_program` and convert all
four tests. These goldens run through no passes, so they are written
directly in post-lowering form. Also correct the module docstring, which
described a `tile.load` + `tile.assemble` lowering when the pass emits
`tile.gather_row` and deliberately no `assemble` (which would lower to an
unsupported MAT->MAT tmov).
Negative facts previously approximated by `not in` greps -- no windowed
clone, no assemble, no Vec tile, no slice of a given parent -- are now
pinned by whole-program equality.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughThe changes strengthen IR transform tests by replacing partial or printed-IR assertions with complete structural comparisons against explicit expected programs. ChangesStructural IR transform tests
Estimated code review effort: 3 (Moderate) | ~25 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary
Twelve tests across three transform test files used a printed-IR substring as their only oracle, leaving everything outside the grepped tokens unchecked. This replaces them with whole-program structural comparisons.
test_auto_tile_matmul_l0.py— the three..._not_foldedguards each asserted the identical pair:Running each test's assertions against the other two tests' output shows all three pass on all three programs — one oracle guarding three distinct rejection reasons (non-matmul consumer /
mode="floor"/ defaultmode="round"), unable to tell them apart. All three are pure no-ops, so this adds_assert_unchanged_by_pass(mirroring the existing helper intest_optimize_orch_tensors.py) and pins the whole program.test_optimize_orch_tensors.py— converts the five remaining substring tests to hand-writtenExpectedgoldens normalized by_run_prereqs_only.Notably,
test_aggregate_output_preserves_existing_pure_input_windowalready carried a complete 67-lineExpectedprogram that was never referenced — the test defined it and then fell through to substring asserts. Nothing flagged it (ruff does not warn on an unused local class). It is now wired up, with the two defects that had stalled it fixed: locals spelled with__(auto-naming rejects those during prerequisite normalization) and a slice order that did not match the pass'sdata/header/outemission order.test_paged_gather.py— had no structural comparison at all. Adds a_build_expectedgolden builder mirroring_build_programand converts all four tests. These goldens run through no passes, so they are written directly in post-lowering form.Negative facts previously approximated by
not ingreps — no windowed clone, noassemble, no Vec tile, no slice of a given parent — are now pinned by whole-program equality.Goldens are never self-referential
No golden is built by running the pass under test:
test_paged_gather.pytest_optimize_orch_tensors.pyOptimizeOrchTensorstest_auto_tile_matmul_l0.pyBeforeDrive-by fix
test_paged_gather.py's module docstring described atile.load+tile.assemblelowering, but the pass emitstile.gather_rowand deliberately noassemble(which would lower to an unsupported MAT→MATpto.tmov). The docstring stated the opposite of what the tests assert; corrected with the reason.Testing
tests/ut/ir/transforms/— 2583 passedtests/ut/full sweep — 8547 passed, 2 skipped (1 pre-existing unrelated failure:test_ir_trace.pyconsole-script smoke test needspip install -e .; verified failing on a clean tree)ruff check/ruff format --checkcleantests/lint/scripts passtranspose, adding a spurious function, renaming a clone, making an unfoldable cast foldable)Related Issues
None.