Skip to content

refactor(parser): TransformContext with explicit lookahead#82

Merged
teezeit merged 1 commit intomainfrom
parser-contract
May 1, 2026
Merged

refactor(parser): TransformContext with explicit lookahead#82
teezeit merged 1 commit intomainfrom
parser-contract

Conversation

@teezeit
Copy link
Copy Markdown
Owner

@teezeit teezeit commented May 1, 2026

Summary

Pure refactor: replaces the implicit nextNode parameter and post-hoc i++ patterns in processNodeList with a first-class TransformContext that every transform receives. Cross-sibling peek/consume is now explicit, auditable, and unit-testable in isolation.

This is the parser-contract step from .github/dev-docs/parser-contract-plan.md (PR #81). It unblocks the bug-fix and syntax-change PRs that follow.

What changed

New: packages/core/src/parser/_context.ts

  • TransformContext interface — peekNext(), consumeNext(), transformChild(), transformChildren(), plus shared helpers (parseAttributes, extractTextContent, isHtmlCommentNode).
  • makeContext(siblings, startIndex, options, deps) — sibling-aware factory used by processNodeList.
  • makeIsolatedContext(options, deps) — for transforms that walk a single child without cross-sibling lookahead.

Modified: packages/core/src/parser/transformer.ts

  • Every transform function migrated from (node, options [, nextNode]) to (node, ctx).
  • processNodeList builds a TransformContext per iteration and advances by reading handle.getCursor().
  • The post-hoc shape rule (if (transformed.type === 'select' && nextNode...) i++;) is gone — consumption now lives next to the peek inside transformParagraph, where the read actually happens.
  • The dropdown lookahead is the only cross-sibling peek in the parser; both single-line and multi-line dropdown branches now call ctx.consumeNext() themselves.
  • New __transformerInternals export (test-only) so isolated parse tests can dispatch through transformNode directly.

New: packages/core/tests/lib/transform-test-helpers.ts

  • mdastFor(markdown) — runs the same MDAST stage that parse() runs, stops before the wiremd transform.
  • makeTestContext(siblings, options) — sibling-aware test context.
  • runTransform(mdast, siblings, options) — fires transformNode and returns { node, cursor } so tests can assert on lookahead consumption.

New: packages/core/tests/parser/dropdown-lookahead.test.ts

  • First per-node parse test. 4 cases pinning the dropdown contract:
    • Dropdown + following list → consumes the list.
    • Dropdown alone → no consume.
    • Dropdown + non-list sibling → no consume.
    • Non-dropdown paragraph + list → no peek.

What this PR is NOT

  • Not a file split. transformer.ts stays in one file. Each function gets the new ctx parameter; that's it.
  • Not a bug fix. All 17 KNOWN_FAILURES continue to fail.
  • Not a syntax change. The user's pending syntax direction (((pill)), :::columns, etc.) lands in separate PRs.

Test plan

  • pnpm typecheck — clean
  • pnpm test — 1,184 passed (1,180 baseline + 4 new isolated parse tests)
  • pnpm review:refresh — "Nothing matched — log unchanged" (zero snapshot drift)
  • All 17 .expected-fail.invariants.ts tests continue failing as expected

🤖 Generated with Claude Code

@teezeit @claude
refactor(parser): TransformContext with explicit lookahead
9829f4d 
Replaces the implicit `nextNode` parameter and post-hoc `i++` patterns
in `processNodeList` with a first-class `TransformContext` that every
transform receives. Cross-sibling peek and consume are now explicit:

- `ctx.peekNext()` / `ctx.consumeNext()` for the dropdown-options
  lookahead in `transformParagraph` (the only cross-sibling peek in
  the parser).
- `ctx.transformChild()` for isolated child recursion (no leak of
  parent siblings into descendants).
- `ctx.transformChildren()` for sibling-aware container children.

The previous post-hoc shape rule in `processNodeList` (consume the
trailing list when the result is `select` or a `container`-with-`select`)
is gone — the consume now lives next to the peek inside
`transformParagraph`, which is where the read happens.

Also adds:
- `packages/core/src/parser/_context.ts` — context interface +
  `makeContext` / `makeIsolatedContext` factories.
- `packages/core/tests/lib/transform-test-helpers.ts` — `mdastFor()`
  and `makeTestContext()` for isolated parse tests.
- `packages/core/tests/parser/dropdown-lookahead.test.ts` — first
  per-node parse test, pinning the dropdown lookahead contract.

Pure refactor: 1,180 existing tests stay green, zero snapshot drift,
all 17 `KNOWN_FAILURES` continue to fail.

See `.github/dev-docs/parser-contract-plan.md` for the full plan.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@teezeit teezeit merged commit cb8c478 into main May 1, 2026
8 checks passed
@teezeit teezeit mentioned this pull request May 1, 2026
Merged
4 tasks
teezeit added a commit that referenced this pull request May 1, 2026
@teezeit @claude
fix(parser): preserve content around opener/closer with trailing whit…
10a991f 
…espace (#83)

## Summary

Fixes 4 bugs in `remark-containers.ts` (2 targeted + 2 incidental) and
recovers content in 2 docs fixtures that was previously silently
dropped.

**Targeted bugs:**
- `::: card ` (trailing spaces on opener) — remark emits a hard
\`break\` node, splitting the opener paragraph into multi-child.
\`collectContainer\` only handled \`children.length === 1\`, so
post-opener content was silently dropped → empty container.
- `::: ` (trailing space on closer) — CASE 1 recognized the closer line
via \`.trim()\` but silently discarded any text after the closer.

**Cascade — incidentally fixed:**
- \`docs/page-layouts/sidebar-layout\` and \`sidebar-with-sections\` had
\`.expected-fail.invariants.ts\` for the same root cause (multi-child
opener paragraph). Both invariants flipped to live.

**Content recovery (no prior expected-fail invariants):**
- \`docs/tabs/basic-tabs\` — \`**Free** — basic plan\` / \`**Pro** —
$12/mo\` lines no longer dropped.
- \`docs/navigation/sidebar-nav\` — \`**App**\` brand line no longer
dropped.

## What changed

**`packages/core/src/parser/remark-containers.ts`**
1. \`collectContainer\` now returns \`{ node, trailing?, nextIndex }\`;
\`processNodes\` and the two recursive callsites push trailing siblings
after the container.
2. CASE 3 has a new \`else if\` branch for multi-child openers —
extracts the post-opener children (skipping a leading hard \`break\` if
present) into a synthetic content paragraph.

**Tests**
- New \`tests/parser/remark-containers-whitespace.test.ts\` — 3 isolated
parse tests pinning the opener/closer whitespace contract (uses the
parser-contract test helpers from PR #82).
- 4 fixture invariants flipped from \`.expected-fail\` to live;
\`KNOWN_FAILURES\` in \`review-gate.test.ts\` trimmed.
- 6 \`REVIEW_LOG.md\` rows ⏳→✅ (HTML output verified by diff inspection
— see note below).

## Known caveat

The React/Tailwind snapshots for \`basic-tabs\` and \`sidebar-nav\`
reveal a **pre-existing** latent bug: \`transformParagraph\` builds
inline \`**strong**\`/\`*em*\` as HTML string literals (lines 776-786 of
transformer.ts), which render correctly in HTML but escape to
\`&lt;strong&gt;\` in React/JSX. This pattern already exists in 10+
other fixture snapshots — not introduced here. Separate PR will land a
proper inline-children fix.

## Test plan

- [x] \`pnpm typecheck\` — clean
- [x] \`pnpm test\` — **1,187 passed** (1,184 baseline + 3 new isolated
remark-containers tests)
- [x] \`pnpm review:refresh\` — 6 fixtures flipped to ⏳ then verified ✅
(HTML output is correct; React/Tailwind escape is a separate latent bug)
- [x] All 4 expected-fail → live invariants pass; no other expected-fail
tests changed

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@teezeit teezeit mentioned this pull request May 2, 2026
Merged
3 tasks
teezeit added a commit that referenced this pull request May 2, 2026
@teezeit @claude
docs: drop unnecessary blank lines before ::: closers + fix cascaded-…
351c654 
…tabs parsing (#87)

## Summary

After PRs #82-86 closed all the closer/blank-line parser bugs, the
trailing blank line that was previously required before every \`:::\`
closer is no longer load-bearing. This PR sweeps it out across all docs
and example fixtures (~50 files, ~500 lines removed) and fixes one
cascaded-tabs parser bug uncovered by the sweep.

## Parser fix — cascaded tabs

Tabs with **no blank line** between adjacent \`::: tab\` containers
(e.g. the \`settings-pattern\` fixture in the docs) was producing a
single tab and dropping the rest.

**Root cause:** the \`text-before-nested-opener\` branch in
\`remark-containers.ts\` was matching on synthetic paragraphs whose
**first line** was itself an opener (\`::: tab Security\\n…\\n:::\\n:::
tab Notifications\`) and pushing the inner content as plain text instead
of recursing into the nested container. That branch's intent is
"paragraph that starts with prose, then has a nested opener inside" —
not "paragraph that's already an opener."

**Fix:** one-line guard — skip \`text-before-nested-opener\` when the
paragraph itself parses as an opener. The next branch (nested container
opener) handles that case correctly via recursion.

## Doc sweep

Removed the trailing blank line before \`:::\` closers in every doc and
example fixture. Pattern: \`/\\n\\n:::(?=\\n|$)/\` → \`\\n:::\`. Net
diff: −922 lines, +416 lines (∆ ~−500 useful blank lines + the parser
change).

## Tests

- [x] 1,206 → 1,207 passing.
- [x] \`settings-pattern\` fixture flips ❌ → ✅. \`KNOWN_FAILURES\` stays
empty.
- [x] All 50 doc/example markdown sweeps verified — no fixture
regressions.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@teezeit