Skip to content

[Rust][Arrow] Coordinate explicit close with recovery - #677

Open
teodordelibasic-db wants to merge 2 commits into
mainfrom
effort/zerobus-sdk-dev-close-recovery
Open

[Rust][Arrow] Coordinate explicit close with recovery#677
teodordelibasic-db wants to merge 2 commits into
mainfrom
effort/zerobus-sdk-dev-close-recovery

Conversation

@teodordelibasic-db

@teodordelibasic-db teodordelibasic-db commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

What changes are proposed in this pull request?

Arrow explicit close previously began supervisor teardown only after its flush attempt completed. If the supervisor was recovering, backoff, connection setup, ready-signal waiting, authentication invalidation, or replay could therefore continue until a separate timeout resolved that phase.

This PR coordinates close and recovery through one level-triggered close state and one supervisor-owned finalization path.

  • Interrupt recovery backoff, reconnect setup, ready-signal waiting, authentication invalidation, and partial replay when close is requested.
  • Keep active ACK processing alive until the close flush completes, then let the supervisor detach the sender and finalize exactly once.
  • Prevent a replacement sender from being published after close teardown begins.
  • Preserve an already-ready ACK, peer error, or reconnect authentication rejection instead of masking it with an older transport error or synthetic flush timeout.
  • Rebuild replay state from the durable watermark and retain only the correct unacknowledged batch suffixes when close interrupts a partial replay.
  • Keep cancelled and repeated close calls resumable and idempotent through one stored outcome.
  • Preserve the pending-relative ACK deadline behavior introduced by [Rust][Arrow] Make ACK deadlines pending-relative #654 when replay and close coordination overlap.

Intentionally deferred

  • Support for concurrent ingest and close across an FFI boundary, which remains documented as unsupported.
  • Fallback finalization after an abnormal supervisor exit or panic.
  • Reuse of the original close deadline after a close future is cancelled and resumed.
  • Post-finalization credential invalidation timing and cleanup.
  • Publication-gap micro-races outside the supported concurrency contract.
  • Replace the private close-cancellation sentinel with typed internal reconnect/commit outcomes.
  • Additional duplicate-waiter and header-provider-specific cancellation coverage.

No public API, FFI signature, or ABI changes are introduced.

How is this tested?

  • test_close_interrupts_recovery_backoff: close interrupts a parked recovery backoff and preserves the triggering error.
  • test_close_interrupts_reconnect_do_put_handshake: close cancels a blocked reconnect DoPut handshake.
  • test_close_interrupts_reconnect_ready_wait: close cancels reconnect ready-signal waiting.
  • test_close_preserves_reconnect_auth_rejection: a reconnect authentication rejection wins over the older transport failure and remains stable across repeated close calls.
  • test_close_during_partial_replay_retains_exact_unacked_suffixes: close during partial replay retains only the durable-watermark suffix and later pending batches.
  • ready_process_error_wins_over_visible_flush_completion: a ready peer error is processed before a simultaneous close-flush completion.
  • ready_ack_work_is_polled_before_visible_flush_completion: ready ACK work is processed before a simultaneous close-flush completion.
  • ready_reconnect_result_wins_over_ready_close_request: a completed reconnect result is not discarded when close becomes ready at the same time.
  • empty_close_target_does_not_mask_terminal_error: closing an empty stream preserves its terminal error.
  • explicit_close_cancellation_finalizes_before_close_publication: a private close cancellation preserves the original recovery error before close-state publication becomes visible.

Fixes #657.

Signed-off-by: teodordelibasic-db <teodor.delibasic@databricks.com>
@teodordelibasic-db teodordelibasic-db self-assigned this Aug 5, 2026
Signed-off-by: teodordelibasic-db <teodor.delibasic@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Rust][Arrow] Coordinate explicit close with recovery

1 participant