feat(builder): add support for block access list (EIP-7928) for flashblocks builder#261
feat(builder): add support for block access list (EIP-7928) for flashblocks builder#261
Conversation
Add access list generation to the flashblocks builder, tracking all state reads and writes during EVM execution indexed by transaction position. This enables parallel execution hints for downstream consumers. Builder-side changes: - Add `access_lists` module to xlayer-builder with types, incremental builder, and DB interceptor (ported from Base's base-access-lists) - Add `process_transaction_state()` to extract access list data from revm changesets — uses `Account.original_info` for pre-state diffing instead of Base's DB wrapper approach (avoids State trait conflicts) - Integrate at all 3 execution points: sequencer txs, cached txs, pool txs - Finalize per-flashblock via `std::mem::take` + `build()` in `build_block()` - Populate `OpFlashblockPayloadMetadata.access_list` with EIP-7928 data - Bump deps/optimism submodule to include op-alloy access_list field 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Stop populating `receipts` and `new_account_balances` in flashblock metadata, replaced by the richer EIP-7928 access list. This aligns with Base PR base/base#1428 which removes these fields post-V1. - Remove `new_account_balances` computation from `build_block()` - Remove `receipts_with_hash` construction from `build_block()` - Set both fields to `None` in all metadata construction sites - Update test framework to handle optional receipts field - Bump deps/optimism submodule with optional metadata fields 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…<>> for FBAL builder Align our flashblock access list implementation 1:1 with Base's base-access-lists crate, and refactor FBALBuilderDb to share its builder with the caller via Arc<Mutex<FlashblockAccessListBuilder>>. Access list crate alignment (types.rs, builder.rs): - Exact 1:1 match with base/base/crates/common/access-lists/src/ - Use revm::primitives::HashMap / HashSet (foldhash-aware) - Use alloy-rlp encoding + keccak256 for fal_hash computation - Remove process_transaction_state helper (unused after refactor) FBALBuilderDb (db.rs) — Arc<Mutex<>> pattern: - Replace owned internal builder with shared Arc<Mutex<FlashblockAccessListBuilder>> - Writes flow directly into caller's builder as txs execute - No finish() + merge() needed — partial progress preserved across early returns (cancellation, fatal errors) - Lock acquired briefly per-commit (single-threaded, no contention) Context.rs integration: - All three execute_* functions use FBALBuilderDb via Base's evm.db_mut().db_mut().method() double-unwrap pattern - set_index() at start, inc_index() after each tx commit - Plain return Ok/Err for early exits — no data loss since access list writes already persisted via shared Arc<Mutex<>> Execution.rs: - ExecutionInfo.access_list_builder is now Arc<Mutex<FlashblockAccessListBuilder>> - derive(Default, Debug) still works via Arc/Mutex blanket impls Builder.rs: - build_block() uses std::mem::take(&mut *lock.guard) to extract accumulated builder and leave an empty default for next flashblock 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
@claude review for me |
XLayer-Bot
left a comment
There was a problem hiding this comment.
The Arc<Mutex<>> shared-builder design is a clean fix for the early-return data-loss bug, and the 1:1 alignment with Base's access-lists crate is a good structural choice. A few things to address before merging.
…ot prefetch hint Combine the per-flashblock EIP-7928 access lists received from the sequencer into a single block-wide `BlockAccessList` covering all transactions up to the current best flashblock revision. The aggregated list is surfaced via `BuildArgs.access_list` and passed to reth's `payload_processor` as a proof-prefetch hint, enabling faster state-root computation during flashblock validation. Implementation details (`cache/raw.rs::access_list_up_to`): - Iterate flashblocks `[0..=up_to]` in ascending order via `BTreeMap::range` - Skip flashblocks missing an access list (rather than aborting) so partial data is preserved - Merge per-address via `HashMap::entry().and_modify().or_insert()` — same address across multiple flashblocks is consolidated into one `AccountChanges` - Concatenate per-field change vectors (`storage_changes`, `balance_changes`, `nonce_changes`, `code_changes`) — safe because tx indices are strictly disjoint across flashblocks by builder construction - Dedupe `storage_reads` per address via `sort_unstable + dedup` - Return `None` when no flashblock in the range carries an access list - Sort final output by address for deterministic ordering 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix stale `FBALBuilderDb::finish` reference in doc comment — `finish()` no longer exists after the `Arc<Mutex<>>` refactor. Updated to point at the `std::mem::take` extraction pattern used at flashblock finalization. - Remove unused `FlashblockAccessListBuilder::merge` — not invoked anywhere since the shared `Arc<Mutex<FlashblockAccessListBuilder>>` design writes directly into a single builder, so merging is unnecessary. `AccountChangesBuilder::merge` is kept (still part of the crate's public surface). - Drop redundant `.iter()` from `info.executed_transactions.iter().len()` in `execute_sequencer_transactions`; use `.len()` directly to match the other two execute_* sites. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
@claude review again |
XLayer-Bot
left a comment
There was a problem hiding this comment.
All previously raised concerns have been addressed:
- Stale
finish()doc comment (resolved): Updated in 6a19b4c to reflect thestd::mem::takeextraction pattern. .iter().len()redundancy (resolved): Fixed to.len()directly in 6a19b4c.- Dead
FlashblockAccessListBuilder::merge(resolved): Removed entirely in 6a19b4c — sharedArc<Mutex<>>design makes merging unnecessary. max_tx_indexoff-by-one (resolved): Confirmed exclusive upper bound matching Base exactly; doc/code mismatch is inherited from Base and harmless.basic()unconditionally inserting emptyAccountChanges(accepted): The author's explanation is technically sound — recording a "touched" entry for pure-read accounts enables downstream payload processor prefetching of basic state proofs (nonce/balance/code_hash) before state root computation. The access list is confirmed to be passed to prewarming paths invalidator.rs. Matches Base's behavior. Accepted.
The Arc<Mutex<FlashblockAccessListBuilder>> shared-builder design cleanly resolves the early-return data-loss bug and the implementation is structurally clean. LGTM.
Introduce `--xlayer.flashblocks-disable-access-list` (default: false) to let operators opt out of EIP-7928 access list consumption on the flashblocks RPC node. Useful for A/B testing, debugging, or comparing state root computation performance with and without prefetch hints. Behavior: - Default (enabled): aggregated access list from incoming flashblocks is forwarded to reth's `payload_processor` as a proof-prefetch hint, speeding up state root computation. - When flag set: `BuildArgs.access_list = None` — payload processor falls back to on-demand proof fetching. Incoming flashblock metadata is still parsed but access list data is ignored at the consumer side. Wiring: - `FlashblocksRpcArgs.flashblocks_disable_access_list` in bin/node/src/args.rs - Threaded through `FlashblocksRpcCtx.disable_access_list` into `RawFlashblocksCache::new(disable_access_list)` - `RawFlashblocksCache::try_get_buildable_args` clears `args.access_list` when disabled, short-circuiting the hint without touching the merge logic inside `access_list_up_to` 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Invert the `if disable_access_list` conditional to `if !disable_access_list` so the primary branch (access list enabled) appears first, matching the default code path. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… args 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
@claude review my latest fixes again |
XLayer-Bot
left a comment
There was a problem hiding this comment.
Two issues found in the new commits, both in the crate. The rest of the refactor — Arc<Mutex<>> shared builder, Base 1:1 alignment, access_list_up_to aggregation, disable flag, and unit tests — looks correct and clean.
Hoist `self.db.commit(changes)` to the outer `DatabaseCommit::commit` so it runs unconditionally. The previous `try_commit` returned early on `self.db.basic(...)?` / `self.db.code_by_hash(...)?` errors before reaching the commit, while the outer wrapper swallowed the error — the EVM thought state was committed but the inner DB diverged, causing subsequent transactions to execute on stale state. Access-list bookkeeping is observational metadata and must not have veto power over EVM-produced state changes. Also remove the unused `AccountChangesBuilder::merge`; the `Arc<Mutex<>>` shared-builder design has no in-tree call site for it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
c94b92a to
a3885eb
Compare
|
@claude review again |
XLayer-Bot
left a comment
There was a problem hiding this comment.
All previously raised concerns have been addressed:
-
State commit safety (, a3885eb): is now called unconditionally in , after runs as a best-effort side-effect. Access-list build errors are logged and never veto a state commit. Verified in .
-
**Dead **: removed entirely in a3885eb — no call sites existed and the shared-builder design has no need for it. Verified in .
No new concerns found in the latest code. The implementation is clean: the three paths all share the builder via , indices are set and incremented correctly, and merges per-flashblock entries with correct deduplication and deterministic ordering. Approving.
|
🔧 CI running — workflow run |
|
✅ XLayer Reth CI passed for commit
|
Cuts the per-flashblock wire payload from ~1.2 MB to ~150-250 KB under stress
by (a) RLP-encoding the EIP-7928 BlockAccessList in the upstream metadata
struct and (b) brotli-compressing every frame on both the WS and P2P pipes.
Wire/protocol changes:
- OpFlashblockPayloadMetadata.access_list becomes Option<Bytes> (RLP-encoded);
new ::new() constructor encodes from a structured BlockAccessList,
block_access_list() lazily decodes on demand
- WS pipe now carries the same Message-shaped, brotli-compressed bytes as the
P2P pipe; producer filters OpBuiltPayload from WS publish
- P2P framing switches from LinesCodec to LengthDelimitedCodec (binary-safe
for brotli output)
- Receiver auto-detects compressed vs legacy uncompressed JSON via leading-`{`
heuristic in compress::try_decompress
Encode-once relay:
- Node::run encodes Message exactly once via frame::encode and reuses the
bytes for both broadcast_message and ws_pub.publish
- BroadcastFrame { bytes, decoded } surfaces both wire bytes and decoded form
on the P2P recv path; relay handlers forward frame.bytes without re-encoding
- WsFrame mirrors this on the RPC WS recv path; persist.rs::handle_relay_flashblocks
forwards frame.bytes straight to downstream WS subscribers
Cache decode-at-insert:
- RawFlashblocksEntry adds block_access_lists: BTreeMap<u64, BlockAccessList>;
insert_flashblock decodes the RLP once, access_list_up_to is allocation-only
over the pre-decoded structures
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The flashblocks WS pipe now ships brotli-compressed `Message` envelopes as binary frames (commit 2e0cf4b), but the in-process `FlashblocksListener` still matched only `Message::Text` and parsed JSON directly, so every frame was dropped and the 5 builder flashblocks tests asserted on 0 captured flashblocks. Decode through `broadcast::frame::decode` so the listener matches the producer's wire format and works for both compressed binary and legacy uncompressed JSON. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…n flashblocks payload (#269) * feat(flashblocks): brotli-compress wire payload + RLP-encode access list Cuts the per-flashblock wire payload from ~1.2 MB to ~150-250 KB under stress by (a) RLP-encoding the EIP-7928 BlockAccessList in the upstream metadata struct and (b) brotli-compressing every frame on both the WS and P2P pipes. Wire/protocol changes: - OpFlashblockPayloadMetadata.access_list becomes Option<Bytes> (RLP-encoded); new ::new() constructor encodes from a structured BlockAccessList, block_access_list() lazily decodes on demand - WS pipe now carries the same Message-shaped, brotli-compressed bytes as the P2P pipe; producer filters OpBuiltPayload from WS publish - P2P framing switches from LinesCodec to LengthDelimitedCodec (binary-safe for brotli output) - Receiver auto-detects compressed vs legacy uncompressed JSON via leading-`{` heuristic in compress::try_decompress Encode-once relay: - Node::run encodes Message exactly once via frame::encode and reuses the bytes for both broadcast_message and ws_pub.publish - BroadcastFrame { bytes, decoded } surfaces both wire bytes and decoded form on the P2P recv path; relay handlers forward frame.bytes without re-encoding - WsFrame mirrors this on the RPC WS recv path; persist.rs::handle_relay_flashblocks forwards frame.bytes straight to downstream WS subscribers Cache decode-at-insert: - RawFlashblocksEntry adds block_access_lists: BTreeMap<u64, BlockAccessList>; insert_flashblock decodes the RLP once, access_list_up_to is allocation-only over the pre-decoded structures 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(flashblocks): decode brotli-compressed WS frames in test listener The flashblocks WS pipe now ships brotli-compressed `Message` envelopes as binary frames (commit 2e0cf4b), but the in-process `FlashblocksListener` still matched only `Message::Text` and parsed JSON directly, so every frame was dropped and the 5 builder flashblocks tests asserted on 0 captured flashblocks. Decode through `broadcast::frame::decode` so the listener matches the producer's wire format and works for both compressed binary and legacy uncompressed JSON. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
EIP-7928 consumers (e.g. reth's `bal_to_hashed_post_state`) extract the post-block value of each account/slot via `Vec::last()`. The FBAL builder produced those vectors via `HashMap<u64, _>::into_iter()`, whose iteration order is non-deterministic, so `.last()` returned an arbitrary mid-block snapshot rather than the entry with the largest tx-index. When the aggregated FBAL is fed into the SR pipeline as a `HashedStateUpdate` (via `payload_processor.spawn(... bal=Some)`), the polluted hashed state diverges from the canonical post-state on heavy blocks, producing a different state root and triggering full flashblocks state cache flushes (`hash_mismatch=true`) on every heavy block. Fix: sort `balance_changes`, `nonce_changes`, `code_changes`, and each per-slot `SlotChanges.changes` ascending by `block_access_index` in `AccountChangesBuilder::build`. Switch `storage_changes` from `drain()` to `into_iter()` for consistency with the other fields and drop the now unnecessary `mut self`. Adds 6 unit tests covering out-of-order insertion across all four change vectors and a round-trip through `FlashblockAccessListBuilder`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tion
Adds 5 verification tests in `cache/raw.rs` that simulate reth's
`bal_to_hashed_post_state` consumer contract on the aggregated
`BlockAccessList` produced by `access_list_up_to`:
* `last_balance_change_resolves_across_three_flashblocks`
* `last_nonce_change_resolves_across_three_flashblocks`
* `same_slot_modified_in_every_flashblock_resolves_to_latest_write`
(the duplicate-SlotChanges case across multiple flashblocks)
* `slot_modified_with_gap_resolves_to_latest_touching_flashblock`
* `multi_slot_each_resolves_independently_across_flashblocks`
These cover gaps the existing tests did not exercise: same slot modified
across multiple flashblocks, multi-tx changes per flashblock, and 3+
flashblock chains. All five pass green, confirming aggregation is
correct under the consumer contract.
Also tightens the `builder.rs` sort tests:
* Drop redundant `.len()` assertion in
`balance_changes_are_sorted_by_block_access_index`.
* Rename `flashblock_builder_round_trip_preserves_per_account_ordering`
-> `flashblock_builder_preserves_per_account_change_ordering`
(no encode/decode happens; the old name was misleading).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Pre-execution writes (EIP-4788 beacon root, EIP-2935 block-hash
history, ...) go directly to `state` via `apply_pre_execution_changes`
and bypass `FBALBuilderDb`, so the wire access list silently misses
them. Reth's `bal_to_hashed_post_state` then builds a `HashedPostState`
without those addresses (Case 3 — system contracts not touched by any
sequencer tx) or with stale parent-state values pulled from
`existing_account` for fields the BAL doesn't override (Case 2 —
addresses where pre-exec changed one field and tx 0 changes another).
Either path produces a state-root divergence vs the canonical block on
heavy blocks.
Adds `FlashblockAccessListBuilder::record_transitions(transitions,
block_access_index)` — a generic recorder that walks
`state.transition_state.transitions` and populates the builder. Logic
mirrors `FBALBuilderDb::update_access_list` exactly:
* balance/nonce: existing acct → record any change; new acct →
record only when non-default (`!is_zero` / `!= 0`).
* code: existing acct → record any code-hash change including
transitions to KECCAK_EMPTY (SELFDESTRUCT-style); new acct →
record only when new hash != KECCAK_EMPTY.
* storage: per-slot `previous_or_original_value != present_value`
→ record `present_value`.
Wired into `execute_pre_steps` at `block_access_index = 0` (collisions
with tx 0's own commits at the same idx resolve naturally — both
target the same `HashMap<u64, _>` key, and tx 0's `commit()` runs
after, so its value wins on overwrite).
Refactor: `execute_sequencer_transactions` now takes `&mut
ExecutionInfo` instead of constructing a fresh one and returning it,
so `execute_pre_steps` can populate `info.access_list_builder` with
pre-exec changes before sequencer txs run.
Adds 8 unit tests covering: prev=Some/new=Some changed/unchanged
paths, code-clear (SELFDESTRUCT-style), prev=None branch with both
default-and-non-default values across two addresses, the no-op input
guards, and the end-to-end pre-exec → tx-0 collision flow through
`AccountChangesBuilder::build` (collision-resolves and
non-collision-preserves cases).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…uilder
Replace the custom FBALBuilderDb implementation with upstream
revm-state's built-in BAL builder integrated into State<DB>. Aligns
with EIP-7928 strict indexing (pre-exec=0, tx K=K+1, post-exec=n+1)
and uses Bal::extend / StorageBal::extend for cross-flashblock merge,
giving BTreeMap-based slot dedup for free. Removes ~695 LoC of custom
BAL code under crates/builder/src/access_lists/.
Also extends raw cache test coverage with three merge-invariant tests
that we depend on but do not directly verify upstream:
- read-then-write across flashblocks promotes a slot to storage_changes
- write-then-read across flashblocks does not demote the prior write
- code_changes from successive flashblocks aggregate with .last()
yielding the highest block_access_index
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ecdee98 to
b08e3e1
Compare
- Add `ExecutionInfo::cumulative_bal` accumulator and `merge_access_list` method, populated per-flashblock in builder via `take_built_bal()` so the per-flashblock metadata still ships the alloy-encoded incremental BAL while the block-wide cumulative BAL is retained for future `OpBuiltPayload` integration. - Switch `RawFlashblocksEntry::block_access_lists` from per-flashblock alloy `BlockAccessList` to revm `Bal`. `try_from_alloy` now runs once per account at insert, and `access_list_up_to` becomes a pure revm-to-revm merge — no bytecode re-decode or `Vec→BTreeMap` rebuild on every query. - Pin EIP-7928 compliance with 21 tests on `merge_access_list`: address lex-sort, slot lex-sort, per-slot bal_index ordering, address dedup, `storage_changes ∩ storage_reads = ∅`, no duplicate index per change list, empty-fields retention for touched-but-unchanged accounts. - Pass `None` for the BAL argument into the StateRootTask pipeline on the FB validator, with a TODO documenting the upstream WIP regressions (PR #23393 small-block hang, PR #23423 heavy-block fragmentation) and the fix landing in PR #23833 — re-enable once upstream stabilizes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Summary
This PR adds EIP-7928 block access list generation to the flashblocks builder and the flashblocks RPC sequence validator.
FBAL Integration on builder
BAL is now published per flashblock via the optional
OpFlashblockPayloadMetadata.access_listfield (okx/optimism#221, ethereum-optimism/optimism#20096), using the upstream revm's BAL builder — opt in viaState::builder().with_bal_builder(), let revm's existingcommit()hook record everything, no custom DB wrapper or parallel data structure.The flashblocks
State<DB>opts in to revm's BAL builder inside the statedb, and every db commit call routes throughbal_state.commit(&changes)first which records account/storage/code/nonce updates at the currentbal_index. Storage reads surface naturally through the same code path.Note that EIP-7928 strict indexing is preserved end-to-end:
bal_index = 0— pre-execution writes (EIP-2935, EIP-4788, Canyon create2-deployer). Afterapply_pre_execution_changes(),state.bump_bal_index()advances to 1 before any sequencer txs run.bal_index = K+1— transaction K's commits. Each of the three execution paths (execute_sequencer_transactions,execute_cached_flashblocks_transactions,execute_best_transactions) callsdb.set_bal_index(executed_count + 1)before the first tx andevm.db_mut().bump_bal_index()after every successful commit. Skipped/invalid txs do NOT bump the index.build_block()extracts the per-FB BAL viastate.take_built_alloy_bal()(which also resetsbal_indexto 0) then re-arms with a freshBal::new(). The next execute_*'sset_bal_index(K+1)restores the global index.FBAL integration on flashblocks RPC
Flashblocks BAL is already supported on #176. BAL support is now accumulated and added on the raw cache, and state accounts are now pre-warmed with the FBAL during flashblocks sequence validation.
NOTE - fbal is intentionally disabled on reth's state root task pipeline (
payload_processor.spawn(... bal=None)). We maintain running the tx execution hashed post state multiproof generation path instead, as BAL is still unstable on the current reth v2.1.0 version.FinishedStateUpdates, causing the sparse trie task to waits timeout. The upstream fix is already in but not on reth v2.1.0 (fix(engine): do not install state hook if BAL is disabled paradigmxyz/reth#23835)TODO
Once BAL integration on the upstream reth stabilizes, we will:
PR #269 builds on top of this branch and introduces wire-protocol breaking changes for BAL integration on flashblocks payload.
Message-shaped, brotli-compressed bytes as the P2P pipe (sent as binary frames);OpBuiltPayloadis filtered from WS publishLinesCodectoLengthDelimitedCodec(binary-safe for brotli output)Note that this is an optimization to reduce the overhead in flashblocks payload packets broadcasted over p2p and websocket connections to RPC nodes (subscribers)
broadcast::frame::{encode,decode}single path, with auto-detect on the receive side for legacy uncompressed JSONBlockAccessListinOpFlashblockPayloadMetadata.access_list(lazy-decoded on demand, decoded-once at cache insert)🤖 Generated with Claude Code