Skip to content

test(ir): Pin transform tests with structural goldens, not IR substrings - #2260

Open
Hzfengsy wants to merge 1 commit into
hw-native-sys:mainfrom
Hzfengsy:test/structural-goldens-for-substring-tests
Open

test(ir): Pin transform tests with structural goldens, not IR substrings#2260
Hzfengsy wants to merge 1 commit into
hw-native-sys:mainfrom
Hzfengsy:test/structural-goldens-for-substring-tests

Conversation

@Hzfengsy

@Hzfengsy Hzfengsy commented Aug 3, 2026

Copy link
Copy Markdown
Member

Summary

Twelve tests across three transform test files used a printed-IR substring as their only oracle, leaving everything outside the grepped tokens unchecked. This replaces them with whole-program structural comparisons.

test_auto_tile_matmul_l0.py — the three ..._not_folded guards each asserted the identical pair:

assert "pl.tile.cast(" in printed
assert "pl.tile.assemble(" not in printed

Running each test's assertions against the other two tests' output shows all three pass on all three programs — one oracle guarding three distinct rejection reasons (non-matmul consumer / mode="floor" / default mode="round"), unable to tell them apart. All three are pure no-ops, so this adds _assert_unchanged_by_pass (mirroring the existing helper in test_optimize_orch_tensors.py) and pins the whole program.

test_optimize_orch_tensors.py — converts the five remaining substring tests to hand-written Expected goldens normalized by _run_prereqs_only.

Notably, test_aggregate_output_preserves_existing_pure_input_window already carried a complete 67-line Expected program that was never referenced — the test defined it and then fell through to substring asserts. Nothing flagged it (ruff does not warn on an unused local class). It is now wired up, with the two defects that had stalled it fixed: locals spelled with __ (auto-naming rejects those during prerequisite normalization) and a slice order that did not match the pass's data/header/out emission order.

test_paged_gather.py — had no structural comparison at all. Adds a _build_expected golden builder mirroring _build_program and converts all four tests. These goldens run through no passes, so they are written directly in post-lowering form.

Negative facts previously approximated by not in greps — no windowed clone, no assemble, no Vec tile, no slice of a given parent — are now pinned by whole-program equality.

Goldens are never self-referential

No golden is built by running the pass under test:

File Golden normalized by
test_paged_gather.py nothing — written in post-lowering form
test_optimize_orch_tensors.py pipeline prefix before OptimizeOrchTensors
test_auto_tile_matmul_l0.py prerequisite lowering only, re-derived from Before

Drive-by fix

test_paged_gather.py's module docstring described a tile.load + tile.assemble lowering, but the pass emits tile.gather_row and deliberately no assemble (which would lower to an unsupported MAT→MAT pto.tmov). The docstring stated the opposite of what the tests assert; corrected with the reason.

Testing

  • tests/ut/ir/transforms/ — 2583 passed
  • tests/ut/ full sweep — 8547 passed, 2 skipped (1 pre-existing unrelated failure: test_ir_trace.py console-script smoke test needs pip install -e .; verified failing on a clean tree)
  • ruff check / ruff format --check clean
  • tests/lint/ scripts pass
  • Goldens mutation-tested — every mutation is caught (flipping transpose, adding a spurious function, renaming a clone, making an unfoldable cast foldable)

Related Issues

None.

Twelve tests across three transform test files used a printed-IR substring
as their only oracle, leaving everything outside the grepped tokens
unchecked. Replace them with whole-program comparisons.

test_auto_tile_matmul_l0.py: the three `..._not_folded` guards asserted the
identical pair `"pl.tile.cast(" in printed` / `"pl.tile.assemble(" not in
printed`, so all three passed on all three programs and could not
distinguish the rejection reason each was written to guard. They are pure
no-ops, so add `_assert_unchanged_by_pass` (mirroring the helper in
test_optimize_orch_tensors.py) and pin the whole program. The golden is a
fresh prerequisite-only lowering of `Before`, so the pass under test never
runs on the right-hand side.

test_optimize_orch_tensors.py: convert the five remaining substring tests to
hand-written `Expected` goldens normalized by `_run_prereqs_only`. One of
them, test_aggregate_output_preserves_existing_pure_input_window, already
carried a complete 67-line `Expected` that was never referenced; wire it up
and fix the two defects that had stalled it (locals spelled with `__`,
which auto-naming rejects during prerequisite normalization, and a slice
order that did not match the pass's data/header/out emission order).

test_paged_gather.py: had no structural comparison at all. Add a
`_build_expected` golden builder mirroring `_build_program` and convert all
four tests. These goldens run through no passes, so they are written
directly in post-lowering form. Also correct the module docstring, which
described a `tile.load` + `tile.assemble` lowering when the pass emits
`tile.gather_row` and deliberately no `assemble` (which would lower to an
unsupported MAT->MAT tmov).

Negative facts previously approximated by `not in` greps -- no windowed
clone, no assemble, no Vec tile, no slice of a given parent -- are now
pinned by whole-program equality.
Copilot AI review requested due to automatic review settings August 3, 2026 09:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 3d9bc208-a769-48c8-8df6-453689eb3823

📥 Commits

Reviewing files that changed from the base of the PR and between 6730e96 and 14c393b.

📒 Files selected for processing (3)
  • tests/ut/ir/transforms/test_auto_tile_matmul_l0.py
  • tests/ut/ir/transforms/test_optimize_orch_tensors.py
  • tests/ut/ir/transforms/test_paged_gather.py

📝 Walkthrough

Walkthrough

The changes strengthen IR transform tests by replacing partial or printed-IR assertions with complete structural comparisons against explicit expected programs.

Changes

Structural IR transform tests

Layer / File(s) Summary
AutoTileMatmulL0 no-op guards
tests/ut/ir/transforms/test_auto_tile_matmul_l0.py
Guard tests compare declined rewrites with fresh prerequisite-only lowering.
Orchestrated tensor transformation goldens
tests/ut/ir/transforms/test_optimize_orch_tensors.py
Optimization tests add expected IR for windowed clones, dynamic readers, loop normalization, parent selection, and aggregate outputs.
Paged-gather lowering goldens
tests/ut/ir/transforms/test_paged_gather.py
Paged-gather tests compare standard, transposed, vector-memory, and dynamic-row lowering with explicit structural goldens.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

Poem

A rabbit checks each node in line,
No hidden cast escapes the sign.
Goldens show the paths just right,
Gather rows land in plain sight.
Structural tests now softly thump—
“No-op means no change!” says the bunny’s jump.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes replacing IR substring assertions with structural golden comparisons in transform tests.
Description check ✅ Passed The description accurately explains the structural golden changes, affected tests, validation, and related documentation correction.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants