Skip to content

feat(executor): lock_timeout + bounded retry for native DDL - #27

Merged
Kiran01bm merged 2 commits into
mainfrom
kiran01bm/lock-timeout-retry-e4
Aug 12, 2026
Merged

feat(executor): lock_timeout + bounded retry for native DDL#27
Kiran01bm merged 2 commits into
mainfrom
kiran01bm/lock-timeout-retry-e4

Conversation

@Kiran01bm

@Kiran01bm Kiran01bm commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

Every transactional native DDL statement now executes under SET LOCAL lock_timeout / statement_timeout with a bounded exponential-backoff retry loop, so a DDL statement can never sit in the lock queue blocking production traffic queued behind it. Only SQLSTATE 55P03 (lock timeout) retries; exhaustion returns a typed BudgetError recording the attempt count.

What

  • pkg/executor/optimistic.go: configurable RetryPolicy (defaults: 3 attempts, 100ms initial backoff, 1s cap), ExecuteNative running each attempt in a fresh transaction with transaction-local timeouts, an injectable sleeper seam for deterministic tests; 57014 (query canceled / statement timeout) deliberately does not retry. Validation rejects zero backoff whenever more than one attempt is configured.
  • internal/cli/cli.go, internal/cli/migrate.go: all existing transactional native execution routed through ExecuteNative; attempts/backoff configurable via flags (1 disables retry) with safe defaults preserved for zero-valued programmatic construction, and parsing tests proving the flags reach the policy.
  • pkg/verdict: refusal verdicts carry attempts (JSON and human output), and the debug log records the exhausted attempt count, so operators and automation can tell an exhausted bounded retry from a single-attempt refusal.
  • Deterministic retry-policy unit tests (backoff curve, cap, exhaustion, cancellation classification, invalid policy) plus a real blocked-DDL integration test proving all attempts are exhausted.

Why

DDL lock acquisition in PostgreSQL queues behind long-running transactions, and everything else then queues behind the DDL — an unbounded wait converts one slow reader into an outage. Transaction-local timeouts (SET LOCAL) cannot leak into pooled connections. CREATE INDEX CONCURRENTLY stays outside the wrapper by design: it cannot run in a transaction, and lock_timeout can cancel its safe snapshot waits and strand an invalid index — it keeps its statement_timeout + invalid-index recovery contract.

Before / after

Before:
  DDL ──▶ waits in lock queue indefinitely
              │
              └ traffic queues behind the waiting DDL

After:
  DDL attempt (tx: SET LOCAL lock_timeout) ──55P03──▶ backoff (exponential, capped)
        │                                                  │
     success                                          attempts left? ──yes──▶ retry
                                                           │
                                                          no ▶ BudgetError (typed)

Every transactional native DDL statement now runs under SET LOCAL
lock_timeout/statement_timeout with a bounded jittered-backoff retry
loop; only SQLSTATE 55P03 lock timeouts retry, exhaustion returns a
typed BudgetError. CREATE INDEX CONCURRENTLY stays outside the wrapper
by design (non-transactional; lock_timeout can cancel its snapshot
waits and strand an invalid index) and keeps its statement_timeout +
invalid-index recovery contract.
Review findings on the bounded-retry path: the flag-to-policy wiring
was untested, zero backoff slipped validation while attempts remained
plural, the AttemptNative wrapper had become a stale-doc delegation
shim, and exhausted attempt counts were invisible to operators and
automation triaging a refusal.
@Kiran01bm
Kiran01bm marked this pull request as ready for review August 12, 2026 04:37
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@morgo morgo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Approved by Morgan's AI agent (moderate-risk bar for pg-sprite): bounded lock_timeout retry is correctly scoped — only SQLSTATE 55P03 retries, each attempt in a fresh transaction so SET LOCAL can't leak through the pool, policy validation rejects unbounded/zero-backoff configs, and the capped-doubling backoff matches its tests. ST-7 target proof still gates every attempt. Nice touch exposing the exhausted attempt count in the verdict JSON for automation.

@Kiran01bm
Kiran01bm merged commit 5123922 into main Aug 12, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants