Skip to content

classify error as retryable when pipe closed in deadlineCappableConn#4567

Merged
jgao54 merged 1 commit into
mainfrom
fix-pipe-error
Jul 15, 2026
Merged

classify error as retryable when pipe closed in deadlineCappableConn#4567
jgao54 merged 1 commit into
mainfrom
fix-pipe-error

Conversation

@jgao54

@jgao54 jgao54 commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Classify a transient error that can happen with DeadlineCapableConn. In #4380 we introduced an in-memory transport between connector and ssh using net.Pipe() to allow it to respect deadlines. This introduced a scenario where if the ssh tunnel is processing read/write when the pipe is closed, we get an io.ErrClosedPipe that ended up unhandled. The error is wrapped in an SSHTunnelClosedError to be precise, to avoid blindly handle all io.ErrClosedPipe that could hide future unrelated bugs.

Fixes: DBI-896

@jgao54
jgao54 requested a review from a team as a code owner July 10, 2026 23:32
@claude

claude Bot commented Jul 10, 2026

Copy link
Copy Markdown

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

@pfcoperez pfcoperez left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

}

if _, ok := errors.AsType[*exceptions.SSHTunnelClosedError](err); ok {
return ErrorRetryRecoverable, ErrorInfo{

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a comment just for my own understanding.

Do these errors need to be notified to telemetry?

From the PR description and the original PR it seems that they only happen in a certain situation where the pipe is going to be terminated anyway.

No action is required from us or the customer. It seems to me that we would only pay attention to them if we were inspecting logs manually, right?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be argued to be overly cautious, but if the error is only to expected to occur once, recoverable allows us to still notice if it persists in case there's a bug in some of our logic. The "pipe closed" refers to the io.Pipe used for SSH, not the ClickPipe, and repeatedly hitting this error on read could mean the ClickPipe becomes blocked.

@jgao54 jgao54 Jul 15, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

echo @ilidemi that this leans overly cautious. Because this net.Pipe() solution is somewhat of an unconventional (and creative) workaround, this provides a peace of mind knowing we'd get notified if error persists (vs. unconditionally ignore it in the Read/Write methods).

}

if _, ok := errors.AsType[*exceptions.SSHTunnelClosedError](err); ok {
return ErrorRetryRecoverable, ErrorInfo{

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be argued to be overly cautious, but if the error is only to expected to occur once, recoverable allows us to still notice if it persists in case there's a bug in some of our logic. The "pipe closed" refers to the io.Pipe used for SSH, not the ClickPipe, and repeatedly hitting this error on read could mean the ClickPipe becomes blocked.

@jgao54
jgao54 enabled auto-merge (squash) July 15, 2026 22:21
@jgao54
jgao54 merged commit bcba5ac into main Jul 15, 2026
18 checks passed
@jgao54
jgao54 deleted the fix-pipe-error branch July 15, 2026 22:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants