fix: retain forester proofs after eligibility ends

[sergeytimoshin]

Summary

• keep the proof-result receiver alive after a forester loses eligibility
• cache proofs that finish late and reuse them in a later eligible slot
• defer duplicate per-tree proof work while that cache is warming
• cover successful and failed late-proof completion with unit tests

Why

The existing cache only drained results that were immediately available when eligibility ended. Proofs still running in the prover completed after the receiver was dropped, so they could not be retried later and the same queue work was recomputed.

Validation

• cargo test -p forester late_proof --lib
• cargo check -p forester
• cargo fmt -p forester -- --check
• git diff --check

Local devnet evidence

Validated with both devnet foresters using TRANSACTION_MAX_CONCURRENT_BATCHES=3. For amt2kaJA14v3urZbZvnc5v2np8jqvc4Z8zDep5wbtzx, one result was available at handoff and three arrived late; all four were retained. The next eligible forester sent the four cached instructions in 1,401 ms without regenerating them.

Transaction: 21oZcBeMdhWTMbFeCdqM5Y7miuF5gUx2z1m6prXE42Hk82DCSY225zaJoGZoKjLrHSdLmMCDRim6qYn4ndsfS2Sr

The queue advanced from start index 977251 to 978251. No forester processing errors were observed; the short indexer-root mismatch afterward was expected lag because on-chain state had already advanced.

Summary by CodeRabbit

• Performance

  • Improved proof processing by collecting late proof results in the background.
  • Avoided duplicate proof work while a tree’s proof cache is warming.
  • Deferred proof generation and prewarming until cache collection is complete.

• Reliability

  • Late proof failures are now handled cleanly, allowing cache warming to finish without storing unsuccessful results.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 35b86da3-09b8-488b-9cc6-70f4b85e4de7

📥 Commits

Reviewing files that changed from the base of the PR and between ad5964f and 55cec09.

📒 Files selected for processing (2)

• forester/src/epoch_manager.rs
• forester/src/processor/v2/tx_sender.rs

Included review availability: Your plan provides up to 2 included reviews per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

Changes

The proof cache now accepts late proof results after TxSender shutdown and remains warming until the proof channel closes. EpochManager skips overlapping V2 proof work and prewarming while the cache is warming.

Proof cache warming coordination

Layer / File(s)	Summary
Late proof collection and warming completion forester/src/processor/v2/tx_sender.rs	TxSender transfers proof receivers to the cache, collects late successes asynchronously, ignores late failures, and completes cache warming when the channel closes. Tests cover both late-success and late-failure paths.
Epoch proof-work warming guards forester/src/epoch_manager.rs	process_light_slot_v2 defers proof work, and prewarm_all_trees_during_wait skips prewarming, when the tree cache is warming.

Priority: ⬇️ Low — Defer this forester proof-cache change because it is a scoped improvement to retain late proofs and avoid duplicate work, with no stated customer or external urgency.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: ⚪ Minimal · up to 55cec

Late proof results are retained for later eligible slots while overlapping proof work is deferred during cache warming. The implemented completion and failure handling is covered, with no current merge-blocking risk identified.

Sequence Diagram(s)

sequenceDiagram
  participant TxSender
  participant ProofChannel
  participant ProofCache
  participant EpochManager
  TxSender->>ProofCache: hand off proof receiver
  ProofChannel-->>ProofCache: deliver late proof results
  ProofCache->>ProofCache: finish warming when channel closes
  EpochManager->>ProofCache: check warming state
  ProofCache-->>EpochManager: return warming state
  EpochManager->>EpochManager: defer overlapping proof work

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 70.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 2 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the main change: retaining forester proofs after eligibility ends so late-completing proofs can be reused.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 2

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch agent/cache-late-proof-results

Warning

Some tools did not complete. Review the errors below.

🔧 Clippy (1.97.1)

Clippy execution failed

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-08T12:15:51.895556Z	55cec09	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 55cec09c00

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Do not consume an eligible slot while the cache warms

When a late-proof collector is still active at the start of an eligible slot, this return reports success, after which process_queue unconditionally clears that scheduled slot at line 1869. The warming period can last up to the prover timeout and can be unbounded while submit_with_backpressure retries queue_full, so an entire eligible slot—and potentially the final eligible slot—can be discarded even if the cached proofs become ready moments later. Poll or wait within the slot until warming completes (or the slot expires) instead of returning successfully here.

Useful? React with 👍 / 👎.