Transactional outbox for guaranteed domain event delivery - #194
Conversation
Persist Gig lifecycle events in the same Redis MULTI/EXEC transaction as aggregate state and indexes. Add a leased outbox relay that publishes to the WebSocket gateway channel, worker queue, and webhook subscribers with stable deduplication keys and retry backoff. Document the Redis architecture and PostgreSQL migration indexes, expose protected relay inspection/trigger endpoints, and cover relay behavior with Supertest and service tests. Closes trustflow-protocol#164
meshackyaro
left a comment
There was a problem hiding this comment.
Solid implementation of the outbox pattern here. A few things worth calling out: committing the outbox write in the same Redis MULTI/EXEC as the Gig state change is the crux of what makes this durable rather than a best-effort side write — get that ordering wrong and you reopen the exact dual-write gap the pattern exists to close. The relay design is thorough too: atomic claiming via Lua avoids double-delivery races between relay instances, and lease reclamation means an instance dying mid-processing doesn't strand events indefinitely. Capped exponential backoff plus surfacing exhausted webhook failures back to the outbox closes the loop so failures are visible rather than silently dropped.
Good call including the PostgreSQL migration doc now rather than after the fact — partial index requirements are much easier to get right when they're designed alongside the Redis version instead of reverse-engineered later. Also appreciate the stable dedup key scheme (gig:<id>:<event-type>) being decided up front, since that's a painful thing to retrofit once consumers depend on it.
Nice cleanup folding in the two strict-mode fixes rather than leaving them for a separate PR. Well done and thanks for your contribution
Summary
Implements #164 using the backend's existing durable state architecture. TrustFlow currently has no PostgreSQL data source; mutable Gig state is stored in Redis, so the outbox is committed in the same Redis
MULTI/EXECtransaction as Gig state and indexes. The included architecture document describes the PostgreSQL migration shape and required partial indexes when a SQL domain store is introduced.Changes
OutboxModulewith durableOutboxService, leasedOutboxRelayService, Redis gateway/queue publisher, and protected operational API.gig.created,gig.accepted,gig.cancelled,gig.expired) in the same atomic Redis transaction as state changes.gig:<id>:<event-type>.trustflow:events:gatewayfor the WebSocket gatewaytrustflow:events:queuefor workersdedupKeyand surface exhausted delivery failures so the outbox can retry.GET /outbox/:idPOST /outbox/relaybackend/TRANSACTIONAL_OUTBOX.mdand relay environment settings.lastErrorcould be read before assignment.Verification
npm run buildpasses.npx jest outbox/outbox.service.spec.ts outbox/outbox.controller.spec.ts gig/gig.service.spec.ts gig/gig.controller.spec.ts gig/gig-expiry-worker.service.spec.ts --runInBandpasses: 60 tests.npm run lint:checkstill reports unrelated existing Prettier violations in Stellar files that this PR does not alter, except the two strict-mode initialization fixes above.Closes #164