fix(service): re-emit stored responses when ProcessDeals dedups retried deals

[0xHansLee]

Problem

When a story→kernel ProcessDeals call times out, the kernel keeps processing and generates the verifier's responses, but they die with the disconnected call. The story-side retry then hits the idempotent-dedup branch, which returned nothing — so the verifier's approvals are lost network-wide. GenerateDeals is retry-safe (it regenerates from sealed coeffs); ProcessDeals was not.

Fix

• Persist each generated response with its deal in one atomic write (AddProcessedDeals into a new EmittedResponses field, kept separate from peers' Responses so replay/rebuild are unaffected). On a retried deal kyber already absorbed, re-emit the stored response instead of dropping it.
• Serialize the DKG-mutating RPCs (ProcessDeals/ProcessResponses/ProcessJustification) per round with a shared dkgMutationMu: a client timeout leaves the abandoned handler running, and its retry (or an adjacent RPC) must not mutate the shared, unsynchronized DistKeyGenerator concurrently. Held across process+persist so the retry re-emits deterministically.

Tests

• TestApplyDeals_ReemitsStoredResponsesOnResubmit
• TestApplyDeals_EmittedResponsesStoredSeparately
• TestApplyDeals_MissingEmittedResponsesSilentlySkips
• TestProcessDeals_ConcurrentRoundSerialized (-race)

issue: #84

[yingyangxu2026]

Verified the fix on a SGX devnet by injecting a ProcessDeals timeout (SIGSTOP the kernel mid-dealing past the story→kernel gRPC deadline); the round recovered black-box, so the re-emit logic works. Two things worth addressing:

The dkgMutationMu serialization doesn't cover all mutators of the shared generator. FinalizeDKG (dkg_finalize.go:107 SetTimeout(), :110 DistKeyShare()) and GenerateDeals (dkg_generate_deals.go:88 Deals()) operate on the same cached *dkg.DistKeyGenerator returned by GetInitDKG/GetResharingNextDKG, but neither takes the mutex — only ProcessDeals/ProcessResponses/ProcessJustification do. Since the whole premise here is that a timed-out handler keeps running, a late/abandoned ProcessResponses (mutating each verifier's aggregator under the lock) can run concurrently with FinalizeDKG's SetTimeout (which also propagates to every aggregator) on the same instance — an unsynchronized data race that can panic or corrupt QUAL. Suggest FinalizeDKG and GenerateDeals also take the per-round dkgMutationMu.

If AddProcessedDeals fails after kyber has already absorbed the deal in memory, the deal is absorbed but EmittedResponses isn't persisted and the cached generator isn't evicted — a retry then hits already-processed, misses on disk, and skips, so that response is lost until cache eviction/restart (a crash actually recovers it via replay, so it's transient). Consider evicting/rebuilding the round's cache on persist failure.

Also: the re-emit branch (dkg_process_deals.go:220-221) emits no log at any level — we grepped the frozen node's journal across all levels during the injection and found nothing for the re-emit path, so there's currently no way to confirm in production that this recovery branch actually fired. A single Info line ("re-emitted stored response for retried deal" with round/code_commitment/sender_index) would make it observable.

[jinn-agent]

The core logic — applyDeals re-emission, dkgMutationMu serialization, EmittedResponses persistence — is sound and the test coverage is thorough. Two issues worth addressing:

1. No defer mu.Unlock() in any of the three handlers (ProcessDeals, ProcessResponses, ProcessJustification). Kyber's ProcessDeal/ProcessResponse/ProcessJustification can panic on malformed input (nil inner fields, unexpected state). A panic inside the locked section leaves the round's dkgMutationMu permanently held, deadlocking all subsequent same-round requests until process restart.

2. Asymmetric eviction on persist failure: applyDeals correctly evicts the cached generator when AddProcessedDeals fails (so the retry can rebuild cleanly). ProcessResponses and ProcessJustification do not: if AddResponses/AddJustifications fails after the generator has already absorbed the items, the retry will find the generator in the already-processed path, silently count them as processed, and call AddResponses again — potentially duplicating entries in the store. This asymmetry is pre-existing for ProcessResponses/ProcessJustification (the mutation lock is new here, but the eviction gap was already present), so it is lower priority. Flagging it since the fix pattern is now established in applyDeals.

---

Review iteration 2 · Commit d4867bb · 2026-07-22T07:24:14Z