Skip to content

secondary: roll back a chunk when the commit fails - #5651

Open
Raed2180416 wants to merge 1 commit into
fluent:masterfrom
Raed2180416:secondary-rollback-write
Open

Raed2180416 wants to merge 1 commit into
fluent:masterfrom
Raed2180416:secondary-rollback-write

Conversation

@Raed2180416

Copy link
Copy Markdown

Which issue(s) this PR fixes:
Fixes #5633

What this PR does / why we need it:

A <secondary> output cannot roll back a chunk. Output#acts_as_secondary defines rollback_write on the secondary to delegate to the primary:

define_method(:rollback_write){ |chunk_id, update_retry: true| @primary_instance.rollback_write(chunk_id, update_retry) }

Output#rollback_write takes update_retry as a keyword argument, so the delegation passes two positional arguments to a method that accepts one, and every call raises ArgumentError: wrong number of arguments (given 2, expected 1).

A delayed commit output calls rollback_write when its commit fails: out_forward does it when an ACK times out or the destination closes the connection. In a <secondary>, the call raises, AckHandler#collect_response logs unexpected error while receiving ack, and the chunk stays in the dequeued state instead of going back to the queue. It is not sent again until shutdown.

The fix passes update_retry as a keyword argument. The chunk is rolled back, the retry state is updated, and the next flush sends the chunk again as it does on the primary.

The new test configures a delayed commit secondary, lets it receive a chunk, and asserts that rollback_write takes the chunk back. It fails at HEAD and passes with the change.

Docs Changes:
None.

Release Note:
A chunk written by a <secondary> output is rolled back when the commit fails, so it is sent again on the next flush; previously the rollback raised ArgumentError and the chunk was not retried.

Output#rollback_write takes update_retry as a keyword argument, but the
delegation defined by acts_as_secondary passed it positionally, so every
call raised ArgumentError and the chunk was never rolled back.

Signed-off-by: Raed <Raed2180416@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 17, 2026 22:48

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@Raed2180416
Raed2180416 force-pushed the secondary-rollback-write branch from 80aa3f6 to a8442b7 Compare November 18, 2026 09:06

@Watson1978 Watson1978 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for fixing the keyword argument. The new test fails against master (7bdf768) and passes on this PR (a8442b7).

The first version of #5636, opened on 2026-11-14 for #5633, made the same one-line change. I reproduced the retry regression from the earlier review with this PR: #5636 (comment)

When a secondary out_forward returns a chunk on an ack failure before the primary's delayed_commit_timeout handles it, the active retry is left unchanged. If this keeps happening, neither retry limit ends the secondary phase, and the same chunk can be sent again before another retry interval has passed. In one 25-second run, no chunks reached the primary after it recovered.

One correction to the earlier review: it says this does not happen with the default heartbeat_type transport. It can still happen there, if the heartbeat enables the node again before the next flush.

I suggest continuing the fix in #5636. Its commit 0440151 also covers an ack failure that arrives after the retry has ended. If this PR moves forward on its own, this regression needs to be fixed before merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A <secondary> output cannot roll back a chunk: rollback_write always raises ArgumentError

3 participants