[py] send extra headers with network.setExtraHeaders

[yuriy-mikityuk]

🔗 Related Issues

Fixes #18116

💥 What does this PR do?

add_extra_header() paused every request with a match-everything intercept and continued it with the header list rebuilt from the beforeRequestSent event. In Chrome that has two consequences: driver.get() hangs until the page load timeout, and headers the event does not list are dropped, so adding one header removes Accept from navigation requests. test_handler_with_classic_navigation already skips Chrome and Edge because interception does not work with classic navigation there, and add_extra_header() put every caller into that state without them asking for interception.

network.setExtraHeaders is in the spec and set_extra_headers() already wraps it, so the registry now keeps the store and pushes it to the browser. The browser adds the headers itself, nothing is paused, and the public API does not change.

🔧 Implementation Notes

The extra-headers intercept, its subscription bookkeeping and the _before_resolve merging are gone, so the registry is about request handlers again. The store stays in the registry because clear_request_handlers() has to keep the headers.

I checked the semantics of the native command in Chrome 154 and Firefox 154: adding a header of the same name replaces the previous one, an empty list clears them, and it composes with request handlers (they see the extra header in beforeRequestSent, and it is still sent when a handler rewrites headers with set_headers).

Verified locally on macOS: bazel test //py:unit passes (35/35), and so do the bidi/network_tests targets for chrome-bidi and firefox-bidi. With the old implementation the new browser test fails on Chrome with a 60 second timeout, which is the hang it guards against.

Cross-binding: .NET and Ruby expose setExtraHeaders directly and have no intercept-based helper, so this brings Python in line with them.

🤖 AI assistance

• No substantial AI assistance used
• AI assisted (complete below)
  • Tool(s): Claude code
  • What was generated: a draft of the change, the tests and this description
  • I reviewed all AI output and can explain the change

💡 Additional Considerations

• Touches BiDi semantics, which AGENTS.md lists as high risk.
• network.setExtraHeaders has to be supported by the browser. It works in Chrome 154 and Firefox 154, but the old implementation only needed interception, so on a browser without the command add_extra_header() now raises. Happy to add a fallback if you would rather keep that working.
• The command is sent without contexts / userContexts, which matches the previous session-wide behaviour. Exposing that scoping could be a follow-up.
• Explicit request handlers combined with driver.get() still hang in Chrome ([🐛 Bug]: network.continueRequest() hangs navigation in Chrome #17373, https://issues.chromium.org/issues/425906330). This PR only stops extra headers from needing interception at all.

🔄 Types of changes

• Bug fix (backwards compatible)

[CLAassistant]

All committers have signed the CLA.

[qodo-code-review]

Code Review by Qodo

Sorry, something went wrong

We weren't able to complete the code review on our side. Please try again manually by commenting /agentic_review on this PR.

[qodo-code-review]

PR Summary by Qodo

[py] Send extra headers through BiDi instead of intercepting requests

🐞 Bug fix 🧪 Tests 🕐 20-40 Minutes

AI Description

• Send extra headers through network.setExtraHeaders to avoid pausing requests and hanging classic
 navigation.
• Preserve browser-generated headers and keep extra headers when request handlers are cleared.
• Add browser and unit coverage for navigation, header removal, and handler composition.

Diagram

graph TD
  API["Extra header API"] --> Registry["Request registry"] --> Store[("Header store")] --> Command["setExtraHeaders"] --> Browser["Browser requests"]

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Fallback to interception on unsupported browsers

• ➕ Preserves extra-header support where network.setExtraHeaders is unavailable.
• ➖ Reintroduces request pausing and the classic-navigation hang on fallback paths.
• ➖ Requires two implementations with different header semantics and additional browser coverage.

Recommendation: Use the native BiDi command as proposed: it fixes the hang and preserves browser-generated headers while retaining the public API. Consider a fallback only if supporting browsers without this command is a requirement; the previous interception approach is not a safe default.

Files changed (4) +80 / -129

Bug fix (1) +15 / -53

_network_handlers.pyPush stored extra headers directly to the browser +15/-53

Push stored extra headers directly to the browser

• Replaces the match-everything intercept and request-header merge with 'network.setExtraHeaders' after each add, remove, or clear. Request-handler subscriptions and reconciliation no longer manage extra headers.

py/private/_network_handlers.py

Tests (2) +58 / -66

network_tests.pyCover classic navigation and header-rewriting handlers +29/-0

Cover classic navigation and header-rewriting handlers

• Adds a 'driver.get()' regression test checking that extra headers arrive without losing 'Accept'. Adds a test confirming extra headers and handler-set headers both reach the server after a handler rewrites request headers.

py/test/selenium/webdriver/common/bidi/network_tests.py

bidi_network_tests.pyVerify native extra-header commands and lifecycle +29/-66

Verify native extra-header commands and lifecycle

• Replaces interception-focused assertions with checks for 'network.setExtraHeaders', case-insensitive names, removal, and clearing. Verifies that handler continuation does not inject extra headers and clearing handlers leaves browser-configured headers intact.

py/test/unit/selenium/webdriver/common/bidi_network_tests.py

Documentation (1) +7 / -10

bidi_enhancements_manifest.pyDocument browser-managed extra headers +7/-10

Document browser-managed extra headers

• Updates the generated Network API documentation to describe native header delivery and no request pausing. Clarifies that clearing request handlers leaves no request-registry intercept but preserves extra headers.

py/private/bidi_enhancements_manifest.py