Skip to content

fix(ingestion): ensure replay idempotency and deterministic dedup - #1304

Open
Nwoyemartha wants to merge 1 commit into
CredenceOrg:mainfrom
Nwoyemartha:fix/1261-replay-idempotency-v2
Open

Nwoyemartha wants to merge 1 commit into
CredenceOrg:mainfrom
Nwoyemartha:fix/1261-replay-idempotency-v2

Conversation

@Nwoyemartha

Copy link
Copy Markdown

Implements durable replay safety and idempotency for Horizon ingestion and reconciliation (issue #1261):

  • SorobanClient: integrate SingleFlight into callRpc() with deterministic operation fingerprinting (SHA-256 of method + canonical JSON params) to coalesce concurrent and retried RPC calls into a single round-trip.
  • horizonBondEvents: add conflicting event detection via resolveConflictingEvent() and ConflictingEventError — when the same event ID arrives with a materially different payload, it is rejected deterministically. Identical replays are safe no-ops that skip business logic but still advance the cursor.
  • ReplayService: wrap each event replay in IdempotentConsumer to prevent duplicate business effects when replayLedgerRange() is re-run after a crash.
  • Add canonicalJson() for deterministic, key-order-independent fingerprinting.
  • Add throwIfCancelled() to SorobanClient for abort-signal fail-fast.

Correctness invariants:

  • Same logical request can be retried safely (SingleFlight coalesces).
  • Duplicate processing results in one committed business effect.
  • Conflicting request-key reuse is rejected (ConflictingEventError).
  • Failed/rejected operations leave no unauthorized partial state.
  • Replay/reordering converges to authoritative state.

Implements durable replay safety and idempotency for Horizon ingestion
and reconciliation (issue CredenceOrg#1261):

- SorobanClient: integrate SingleFlight into callRpc() with deterministic
  operation fingerprinting (SHA-256 of method + canonical JSON params) to
  coalesce concurrent and retried RPC calls into a single round-trip.
- horizonBondEvents: add conflicting event detection via resolveConflictingEvent()
  and ConflictingEventError — when the same event ID arrives with a materially
  different payload, it is rejected deterministically. Identical replays are
  safe no-ops that skip business logic but still advance the cursor.
- ReplayService: wrap each event replay in IdempotentConsumer to prevent
  duplicate business effects when replayLedgerRange() is re-run after a crash.
- Add canonicalJson() for deterministic, key-order-independent fingerprinting.
- Add throwIfCancelled() to SorobanClient for abort-signal fail-fast.

Correctness invariants:
  - Same logical request can be retried safely (SingleFlight coalesces).
  - Duplicate processing results in one committed business effect.
  - Conflicting request-key reuse is rejected (ConflictingEventError).
  - Failed/rejected operations leave no unauthorized partial state.
  - Replay/reordering converges to authoritative state.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant