CDP: Page.navigatedWithinDocument is never emitted for same-document navigations (history API / fragment / traversal)

[bebsworthy]

Summary

Lightpanda performs same-document navigations correctly in-page (history.pushState/replaceState update window.location; fragment changes and popstate work), but it never emits the CDP Page.navigatedWithinDocument event. As a result, any CDP client (Playwright/Puppeteer) never learns about in-document URL changes: the frame's tracked URL goes stale and SPA route changes are invisible to automation. Full-document navigations are unaffected (they correctly emit Page.frameNavigated).

Environment

• lightpanda serve, driven over CDP. Reproduced on 1.0.0-dev; the gap is present at current HEAD.
• Client: Playwright 1.59 via chromium.connectOverCDP('ws://127.0.0.1:9222').

Expected vs actual

A real Chromium fires Page.navigatedWithinDocument { frameId, url } whenever a same-document navigation happens — history.pushState/replaceState, history.back/forward/go within the same document, fragment (#) navigation, and same-document anchor clicks. Playwright uses this to update frame.url() / page.url().

Lightpanda: the in-page location updates, but no CDP event is sent, so page.url() stays at the pre-pushState value indefinitely, and expect(page).toHaveURL(...) / page.waitForURL(...) never observe the change.

Minimal repro

await page.goto('http://x.test/a');                       // mock via page.route
await page.evaluate(() => history.pushState({}, '', '/b'));
// Expected: Page.navigatedWithinDocument fires, page.url() === 'http://x.test/b'
// Actual:   no event; location.pathname is '/b' (in-page OK) but page.url() stays '/a'

Same result for location.hash = '#x' and for history.back() after a pushState.

Real-world impact

SPA frameworks (e.g. react-router v7 createBrowserRouter) route client-side via history.pushState. Because the route change is never reported over CDP, automated tests can't assert on the URL after navigation. Anchor-click navigation itself works (it does an async full-document load via scheduleNavigation, which emits Page.frameNavigated); the gap is specifically the same-document path.

Where it's missing in the source

Three code paths change frame.url in place without a document reload, and none emit a CDP event (grep -rn navigatedWithinDocument src/ returns nothing):

1. src/browser/webapi/History.zig — pushState / replaceState set frame.url + reinit window._location.
2. src/browser/Frame.zig — the same-document fragment short-circuit in scheduleNavigationWithArena (anchor clicks, location.hash).
3. src/browser/webapi/navigation/Navigation.zig — navigateInner's same-document .push / .replace / .traverse branches (the Navigation API and history.back/forward/go).

By contrast, full-document navigations dispatch .frame_navigated → src/cdp/domains/page.zig:frameNavigated → Page.frameNavigated. There is no same-document analogue.

CDP shape to emit

Page.navigatedWithinDocument
  frameId: <main frame id, same one used for Page.frameNavigated>
  url: <new absolute URL>
  navigationType: "historyApi" | "fragment" | "other"

Implementation

I have a fix (with CDP-level regression tests, full suite green):
bebsworthy@8a51a0b
(branch: fix/cdp-navigated-within-document)

A new Notification.FrameNavigatedWithinDocument event (frame_id, url, navigation_type) is dispatched from the three paths above via a Frame.notifyNavigatedWithinDocument helper and handled in page.zig, which emits Page.navigatedWithinDocument { frameId, url, navigationType }. Unlike frameNavigated, it does not clear/recreate the execution context or send DOM.documentUpdated — the document is unchanged, matching Chrome. navigationType maps to historyApi (pushState/replaceState, Navigation API, traversal) or fragment (anchor / location.hash).

Related issues

• Click/submit on Amazon PDP does not trigger navigation (works in Chromium) #1401 (open) — programmatic click runs but page.url() unchanged over CDP (closest existing match).
• Question: Roadmap for CDP automation features (setFileInputFiles, Input events, dialog, history, window.open)? #2043 (open) — CDP automation roadmap; lists "pushState updates location" but tracks no navigatedWithinDocument event.
• Are there currently no implementations for SPAs like React/Vue that require a complete JS runtime? #1798 (open) — SPA/Web-API support in progress.
• update page URL and location on pushState/replaceState #2131 (merged) — made pushState update location JS-side, but added no CDP event (the JS-side precedent this builds on).

[rohitsux]

Opened #2964 implementing the History-API slice of this (pushState/replaceState -> Page.navigatedWithinDocument), with a cdp.frame test. @bebsworthy I saw you have a fix on a fork that isn't upstreamed yet — I didn't want this to stall, but happy to defer to your branch or fold in your approach; and the fragment / Navigation-API paths are open as follow-ups if you'd rather take those. Whatever the maintainers prefer.