feat(l1): engine REST/SSZ API#6710
Conversation
Adds the binary SSZ engine API alongside JSON-RPC on the authrpc port, per execution-apis PR #764. JSON-RPC remains the default; supported endpoints are advertised via engine_exchangeCapabilities as strings like "POST /engine/v5/payloads" so the CL can opt into the binary transport per endpoint. Endpoints (version corresponds to engine_*V{N} JSON-RPC method version): POST /engine/v{1..5}/payloads newPayload GET /engine/v{1..6}/payloads/{payload_id} getPayload POST /engine/v{1,2}/payloads/bodies/by-hash getPayloadBodiesByHash POST /engine/v{1,2}/payloads/bodies/by-range getPayloadBodiesByRange POST /engine/v{1..4}/forkchoice forkchoiceUpdated POST /engine/v{1..3}/blobs getBlobs POST /engine/v1/client/version getClientVersion POST /engine/v1/capabilities exchangeCapabilities Surface lives under `crates/networking/rpc/engine_rest/`: - `types/`: SSZ wire types for every container in #764 (ExecutionPayloadV1..V4, BlobsBundleV1/V2, ForkchoiceState, payload attributes, payload bodies, blob requests/responses, capabilities, client version) plus shared constants and `SszOption<T> = SszList<T,1>` for SSZ's `Option<T>` encoding. - `handlers/`: per-endpoint axum handlers. - `conversions.rs`: SSZ <-> ethrex internal type conversions. New direct SSZ payload -> Block decoders skip the JsonExecutionPayload intermediate on the incoming newPayload path; per-tx allocations go from two to one. - `auth.rs`: JWT bearer middleware (shared secret with JSON-RPC). - `extractors.rs`: `Ssz<T>` axum extractor + Content-Type guard. - `responses.rs`: `SszBody` 200-OK wrapper with `application/octet-stream`. - `observe.rs`: per-request metrics + warn-on-error middleware feeding the same `rpc_request_duration_seconds` / `rpc_requests_total` series JSON-RPC uses. `error_kind` labels are carried in `EngineErrorContext` so they match the JSON-RPC vocabulary (`UnsupportedFork`, `WrongParam`, ...). - `error.rs`: typed `EngineRestError` (status + message + error_kind) with `From<RpcErr>` mapping to consistent HTTP status codes (404 for UnknownPayload, 409 for InvalidForkChoiceState/TooDeepReorg, 422 for UnsupportedFork/InvalidPayloadAttributes, 413 for TooLargeRequest, 401 for auth failure, 400 for malformed params, 500 otherwise). - `tests.rs`: end-to-end tower-oneshot tests covering JWT, content-type rejection, capability advertisement, blobs empty mempool, bodies unknown-hash, forkchoice_v1 / new_payload_v1 status codes, payload-id parse errors, and pre-Shanghai newPayloadV2 with/without withdrawals. Also adds: - EIP-7783 `target_gas_limit` plumbed through `BuildPayloadArgs`. The 0-sentinel ("unset") collapse happens in `build_payload_v4` so JSON-RPC and SSZ converge on the same args and payload ID. JSON-RPC's `PayloadAttributesV4` gains `Option<u64>` via `hex_str_opt`. - The JSON-RPC `handle_new_payload_v{1,2,3,4}` signatures take `expected_block_hash: H256` directly instead of `&ExecutionPayload`; callers updated. - `ExecutionPayload::into_block` -> `to_block(&self, ...)` to avoid the payload clone on the JSON-RPC newPayload path. - engine_exchangeCapabilities (JSON-RPC) now advertises SSZ REST endpoints alongside method names. Fork gating + correctness: - newPayloadV2 rejects withdrawals on pre-Shanghai chains. - getBlobsV{2,3} require Osaka tip. - forkchoiceV1/V2 reject building Cancun payloads. - bodies_by_range_v{1,2} guard `start + count - 1` against overflow via `saturating_add(...).min(latest)` plus a `start > latest` short-circuit. - V2 body handlers run header lookup + EVM-bound BAL generation inside a single `spawn_blocking` per request (BAL re-executes the block); bodies_by_hash_v2 / bodies_by_range_v2 share `assemble_blocks_with_bal`. - client_version returns [0u8;4] on non-hex commit instead of 500. - payload_id parse uses a typed `PayloadIdParseError`. Limits: SSZ-side caps (`MAX_PAYLOAD_BODIES_REQUEST = 32`, etc.) match #764 spec values; JSON-RPC remains permissive (32 floor / up to 128). The body-limit asymmetry is documented on both constants. Per-endpoint Content-Length caps from #764 §Security considerations are not enforced; the authrpc router's global 256 MB DefaultBodyLimit covers both transports. Bench (`benches/engine_transport.rs`): - direct SSZ -> Block decode ~28us vs JSON-intermediate ~33us per 150-tx payload (~17% faster, ~80 KB less per request). - `blobs_bundle_to_ssz_v2` 6-blob bundle drops from ~10us to ~7ns by taking the bundle by value and moving Vecs. Workspace fallout: `BuildPayloadArgs { target_gas_limit: None }` added to four blockchain integration tests and one bench. `Cargo.lock` / `Cargo.toml` pull `libssz`, `libssz-derive`, `libssz-types`, `futures`, and a dev-only `tower`+`criterion` for the new bench.
|
🤖 Kimi Code ReviewThis PR implements SSZ REST endpoints for the Engine API (per execution-apis PR #764) and EIP-7783 target gas limit support. The implementation is generally solid, well-tested, and follows Rust best practices. Critical Issues
Correctness & Safety
Performance & Idioms
Nits
Summary The PR is well-structured with good test coverage (unit tests for payload ID calculation, integration tests for the REST router, and benchmarks). Fix Item 1 (the off-by-one check) before merging. The SSZ type safety and JWT auth integration are implemented correctly. Automated review by Kimi (Moonshot AI) · kimi-k2.5 · custom prompt |
Lines of code reportTotal lines added: Detailed view |
🤖 Codex Code Review
I did not run the Rust tests successfully here: Automated review by OpenAI Codex · gpt-5.4 · custom prompt |
Greptile SummaryThis PR adds a REST/SSZ transport for the Engine API (per execution-apis #764), introducing binary SSZ endpoints under
Confidence Score: 3/5The JWT auth wiring, SSZ encode/decode plumbing, and middleware ordering are all correct, but the off-by-one in the blob hash count guard will cause any CL request with exactly 128 blob hashes to be rejected with a 413 instead of served normally. The blob check_count uses >= where it should use >, meaning a full-sized request of 128 hashes is incorrectly rejected. Because blob data is needed by the CL for block validation, this could interrupt normal operation for clients that happen to request the maximum count. All other handler logic and the refactored engine core functions look correct. crates/networking/rpc/engine_rest/handlers/blobs.rs -- the check_count guard and the optional Osaka check in require_osaka_tip
|
| Filename | Overview |
|---|---|
| crates/networking/rpc/engine_rest/handlers/blobs.rs | New getBlobs V1/V2/V3 SSZ handlers; check_count uses >= instead of > which rejects exactly 128 hashes (the allowed maximum) |
| crates/networking/rpc/engine_rest/handlers/payloads.rs | New SSZ newPayload/getPayload handlers V1-V6; logic mirrors JSON-RPC path with correct fork validation checks |
| crates/networking/rpc/engine_rest/handlers/forkchoice.rs | New SSZ forkchoiceUpdated V1-V4 handlers; SSZ to internal type conversions and build_payload flow are correct |
| crates/networking/rpc/engine_rest/handlers/bodies.rs | SSZ getPayloadBodies V1/V2 handlers; check_count correctly uses > for the bodies cap; BAL via spawn_blocking is correct |
| crates/networking/rpc/engine_rest/conversions.rs | SSZ to internal type conversions; one misleading error message re BAL using MAX_BYTES_PER_TRANSACTION label |
| crates/networking/rpc/engine_rest/auth.rs | JWT Bearer auth middleware shared with JSON-RPC; correct implementation |
| crates/networking/rpc/engine_rest/mod.rs | Engine REST router construction; auth and observe middleware are correctly ordered (auth outermost) |
| crates/networking/rpc/engine_rest/error.rs | EngineRestError and RpcErr to HTTP status mapping; clean implementation with proper status codes |
| crates/networking/rpc/engine/payload.rs | Refactored to accept H256 instead of &ExecutionPayload for block hash validation; functions made pub(crate) for SSZ layer reuse |
| test/tests/rpc/engine_rest_tests.rs | Integration tests covering JWT auth, capabilities, newPayload, getPayload, bodies, blobs, and forkchoice round-trips |
| crates/networking/rpc/engine_rest/observe.rs | Metrics/warn middleware mapping HTTP method+path to JSON-RPC method labels; correctly placed inside auth layer |
| crates/networking/rpc/engine_rest/types/common.rs | Fork-invariant SSZ types and constants; nullable encoding helpers are correct |
Sequence Diagram
sequenceDiagram
participant CL as Consensus Layer
participant AM as JWT Auth Middleware
participant OM as Observe Middleware
participant RH as REST Handler (SSZ)
participant EC as Engine Core (pub(crate))
participant ST as Storage / Mempool
CL->>AM: POST /engine/vN/... (Bearer JWT + SSZ body)
AM->>AM: validate_jwt_authentication()
alt Invalid JWT
AM-->>CL: 401 Unauthorized (text/plain)
end
AM->>OM: forward request
OM->>RH: forward request
RH->>RH: Ssz extractor (Content-Type check + SszDecode)
alt Bad Content-Type or malformed SSZ
RH-->>CL: 400 Bad Request
end
RH->>EC: handle_new_payload / handle_forkchoice / get_payload
EC->>ST: read/write block data
ST-->>EC: Result
EC-->>RH: PayloadStatus / PayloadBundle
RH->>RH: SszEncode response
RH-->>OM: Response (200 + application/octet-stream)
OM->>OM: record_rpc_outcome / warn on non-2xx
OM-->>CL: SSZ response
Prompt To Fix All With AI
Fix the following 3 code review issues. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 3
crates/networking/rpc/engine_rest/handlers/blobs.rs:24-31
Off-by-one in blob hash count guard: the condition `n >= MAX_BLOB_HASHES_REQUEST` rejects requests that contain exactly 128 hashes, even though 128 is the defined maximum. Compare with `bodies.rs` which correctly uses `n > MAX_PAYLOAD_BODIES_REQUEST`. A CL sending exactly 128 blob hashes will get a 413 instead of a 200.
```suggestion
fn check_count(n: usize) -> Result<(), EngineRestError> {
if n > MAX_BLOB_HASHES_REQUEST {
return Err(EngineRestError::payload_too_large(format!(
"request exceeds MAX_BLOB_HASHES_REQUEST ({MAX_BLOB_HASHES_REQUEST})"
)));
}
Ok(())
}
```
### Issue 2 of 3
crates/networking/rpc/engine_rest/handlers/blobs.rs:85-98
**Silent pass-through when no genesis header exists**
`require_osaka_tip` returns `Ok(())` when `get_block_header(latest)` returns `None` (e.g. on a freshly-initialized in-memory store before any block is written). In that state `blobs_v2` and `blobs_v3` would be served without verifying Osaka activation, so a non-Osaka chain that hasn't yet written a genesis header would incorrectly serve these endpoints instead of returning 422.
### Issue 3 of 3
crates/networking/rpc/engine_rest/conversions.rs:184-186
The error message says "BAL RLP exceeds MAX_BYTES_PER_TRANSACTION" when the value being bounded is the Block Access List, not a transaction. The numeric cap itself is correct (the SSZ field type is `SszList<u8, MAX_BYTES_PER_TRANSACTION>`) but the label will confuse anyone reading logs or error responses.
```suggestion
let bal_ssz = bal_bytes
.try_into()
.map_err(|_| ConversionError::internal("BAL RLP exceeds MAX_BYTES_PER_TRANSACTION (block_access_list field)"))?;
```
Reviews (1): Last reviewed commit: "feat(rpc): engine REST/SSZ API per execu..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| fn check_count(n: usize) -> Result<(), EngineRestError> { | ||
| if n >= MAX_BLOB_HASHES_REQUEST { | ||
| return Err(EngineRestError::payload_too_large(format!( | ||
| "request exceeds MAX_BLOB_HASHES_REQUEST ({MAX_BLOB_HASHES_REQUEST})" | ||
| ))); | ||
| } | ||
| Ok(()) | ||
| } |
There was a problem hiding this comment.
n >= MAX_BLOB_HASHES_REQUEST rejects requests that contain exactly 128 hashes, even though 128 is the defined maximum. Compare with bodies.rs which correctly uses n > MAX_PAYLOAD_BODIES_REQUEST. A CL sending exactly 128 blob hashes will get a 413 instead of a 200.
| fn check_count(n: usize) -> Result<(), EngineRestError> { | |
| if n >= MAX_BLOB_HASHES_REQUEST { | |
| return Err(EngineRestError::payload_too_large(format!( | |
| "request exceeds MAX_BLOB_HASHES_REQUEST ({MAX_BLOB_HASHES_REQUEST})" | |
| ))); | |
| } | |
| Ok(()) | |
| } | |
| fn check_count(n: usize) -> Result<(), EngineRestError> { | |
| if n > MAX_BLOB_HASHES_REQUEST { | |
| return Err(EngineRestError::payload_too_large(format!( | |
| "request exceeds MAX_BLOB_HASHES_REQUEST ({MAX_BLOB_HASHES_REQUEST})" | |
| ))); | |
| } | |
| Ok(()) | |
| } |
Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/networking/rpc/engine_rest/handlers/blobs.rs
Line: 24-31
Comment:
Off-by-one in blob hash count guard: the condition `n >= MAX_BLOB_HASHES_REQUEST` rejects requests that contain exactly 128 hashes, even though 128 is the defined maximum. Compare with `bodies.rs` which correctly uses `n > MAX_PAYLOAD_BODIES_REQUEST`. A CL sending exactly 128 blob hashes will get a 413 instead of a 200.
```suggestion
fn check_count(n: usize) -> Result<(), EngineRestError> {
if n > MAX_BLOB_HASHES_REQUEST {
return Err(EngineRestError::payload_too_large(format!(
"request exceeds MAX_BLOB_HASHES_REQUEST ({MAX_BLOB_HASHES_REQUEST})"
)));
}
Ok(())
}
```
How can I resolve this? If you propose a fix, please make it concise.| .storage | ||
| .get_block_header(latest) | ||
| .map_err(|e| EngineRestError::internal(format!("storage: {e}")))?; | ||
| if let Some(h) = header | ||
| && !ctx | ||
| .storage | ||
| .get_chain_config() | ||
| .is_osaka_activated(h.timestamp) | ||
| { | ||
| return Err(EngineRestError::unprocessable(format!( | ||
| "getBlobsV{version} engine only supported for Osaka" | ||
| ))); | ||
| } | ||
| Ok(()) |
There was a problem hiding this comment.
require_osaka_tip returns Ok(()) when get_block_header(latest) returns None (e.g. on a freshly-initialized in-memory store before any block is written). In that state blobs_v2 and blobs_v3 would be served without verifying Osaka activation, so a non-Osaka chain that hasn't yet written a genesis header would incorrectly serve these endpoints instead of returning 422.
Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/networking/rpc/engine_rest/handlers/blobs.rs
Line: 85-98
Comment:
**Silent pass-through when no genesis header exists**
`require_osaka_tip` returns `Ok(())` when `get_block_header(latest)` returns `None` (e.g. on a freshly-initialized in-memory store before any block is written). In that state `blobs_v2` and `blobs_v3` would be served without verifying Osaka activation, so a non-Osaka chain that hasn't yet written a genesis header would incorrectly serve these endpoints instead of returning 422.
How can I resolve this? If you propose a fix, please make it concise.| let bal_ssz = bal_bytes | ||
| .try_into() | ||
| .map_err(|_| ConversionError::internal("BAL RLP exceeds MAX_BYTES_PER_TRANSACTION"))?; |
There was a problem hiding this comment.
SszList<u8, MAX_BYTES_PER_TRANSACTION>) but the label will confuse anyone reading logs or error responses.
| let bal_ssz = bal_bytes | |
| .try_into() | |
| .map_err(|_| ConversionError::internal("BAL RLP exceeds MAX_BYTES_PER_TRANSACTION"))?; | |
| let bal_ssz = bal_bytes | |
| .try_into() | |
| .map_err(|_| ConversionError::internal("BAL RLP exceeds MAX_BYTES_PER_TRANSACTION (block_access_list field)"))?; |
Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/networking/rpc/engine_rest/conversions.rs
Line: 184-186
Comment:
The error message says "BAL RLP exceeds MAX_BYTES_PER_TRANSACTION" when the value being bounded is the Block Access List, not a transaction. The numeric cap itself is correct (the SSZ field type is `SszList<u8, MAX_BYTES_PER_TRANSACTION>`) but the label will confuse anyone reading logs or error responses.
```suggestion
let bal_ssz = bal_bytes
.try_into()
.map_err(|_| ConversionError::internal("BAL RLP exceeds MAX_BYTES_PER_TRANSACTION (block_access_list field)"))?;
```
How can I resolve this? If you propose a fix, please make it concise.
🤖 Claude Code ReviewNow I have a comprehensive picture of the PR. Let me compile the review. PR #6710 Review:
|
1 participant
Motivation
We want to reduce the overhead of JSON serialization/deserialization for the large objects handled by the engine API. One of the proposals to do so is ethereum/execution-apis#764
Description
Most of the PR consists of boilerplate: defining RPC objects, route handlers and serialization/deserialization logic.