Skip to content

Transactional outbox for guaranteed domain event delivery - #194

Merged
meshackyaro merged 3 commits into
trustflow-protocol:mainfrom
xqcxx:feat/transactional-outbox
Aug 27, 2026
Merged

Transactional outbox for guaranteed domain event delivery#194
meshackyaro merged 3 commits into
trustflow-protocol:mainfrom
xqcxx:feat/transactional-outbox

Conversation

@xqcxx

@xqcxx xqcxx commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Summary

Implements #164 using the backend's existing durable state architecture. TrustFlow currently has no PostgreSQL data source; mutable Gig state is stored in Redis, so the outbox is committed in the same Redis MULTI/EXEC transaction as Gig state and indexes. The included architecture document describes the PostgreSQL migration shape and required partial indexes when a SQL domain store is introduced.

Changes

  • Added OutboxModule with durable OutboxService, leased OutboxRelayService, Redis gateway/queue publisher, and protected operational API.
  • Persists Gig lifecycle events (gig.created, gig.accepted, gig.cancelled, gig.expired) in the same atomic Redis transaction as state changes.
  • Stable consumer dedup keys: gig:<id>:<event-type>.
  • Relay claims due events atomically using Lua, leases processing rows, reclaims abandoned leases, and retries failed deliveries with exponential backoff capped at 30 seconds.
  • Relays to:
    • Redis pub/sub channel trustflow:events:gateway for the WebSocket gateway
    • Redis list trustflow:events:queue for workers
    • registered webhook subscribers
  • Webhooks now receive dedupKey and surface exhausted delivery failures so the outbox can retry.
  • Added authenticated OpenAPI-documented endpoints:
    • GET /outbox/:id
    • POST /outbox/relay
  • Added backend/TRANSACTIONAL_OUTBOX.md and relay environment settings.
  • Migrated existing direct Gig webhook dispatches to the outbox relay.
  • Fixed two existing strict TypeScript build blockers where lastError could be read before assignment.

Verification

  • npm run build passes.
  • Targeted ESLint passes for all changed production and test files.
  • npx jest outbox/outbox.service.spec.ts outbox/outbox.controller.spec.ts gig/gig.service.spec.ts gig/gig.controller.spec.ts gig/gig-expiry-worker.service.spec.ts --runInBand passes: 60 tests.
  • npm run lint:check still reports unrelated existing Prettier violations in Stellar files that this PR does not alter, except the two strict-mode initialization fixes above.

Closes #164

Persist Gig lifecycle events in the same Redis MULTI/EXEC transaction as
aggregate state and indexes. Add a leased outbox relay that publishes to the
WebSocket gateway channel, worker queue, and webhook subscribers with stable
deduplication keys and retry backoff.

Document the Redis architecture and PostgreSQL migration indexes, expose
protected relay inspection/trigger endpoints, and cover relay behavior with
Supertest and service tests.

Closes trustflow-protocol#164
@xqcxx
xqcxx requested a review from meshackyaro as a code owner August 26, 2026 13:39

@meshackyaro meshackyaro left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid implementation of the outbox pattern here. A few things worth calling out: committing the outbox write in the same Redis MULTI/EXEC as the Gig state change is the crux of what makes this durable rather than a best-effort side write — get that ordering wrong and you reopen the exact dual-write gap the pattern exists to close. The relay design is thorough too: atomic claiming via Lua avoids double-delivery races between relay instances, and lease reclamation means an instance dying mid-processing doesn't strand events indefinitely. Capped exponential backoff plus surfacing exhausted webhook failures back to the outbox closes the loop so failures are visible rather than silently dropped.

Good call including the PostgreSQL migration doc now rather than after the fact — partial index requirements are much easier to get right when they're designed alongside the Redis version instead of reverse-engineered later. Also appreciate the stable dedup key scheme (gig:<id>:<event-type>) being decided up front, since that's a painful thing to retrofit once consumers depend on it.

Nice cleanup folding in the two strict-mode fixes rather than leaving them for a separate PR. Well done and thanks for your contribution

@meshackyaro
meshackyaro merged commit afab01e into trustflow-protocol:main Aug 27, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement a Transactional Outbox for Guaranteed Event Delivery

2 participants