Skip to content

feat(storage): add PostgreSQL dialect nucleus to the shared store core - #1010

Merged
Kiran01bm merged 10 commits into
mainfrom
kiran01bm/pg-backend-nucleus
Aug 13, 2026
Merged

feat(storage): add PostgreSQL dialect nucleus to the shared store core#1010
Kiran01bm merged 10 commits into
mainfrom
kiran01bm/pg-backend-nucleus

Conversation

@Kiran01bm

@Kiran01bm Kiran01bm commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds the PostgreSQL nucleus to the shared sqlstore core: a PostgresDialect implementing the binder, SQL-rendering, and identity seams that the dialect-parameterized store already consumes, plus the first cross-dialect parity lanes proving the shared SQL runs unmodified on PostgreSQL.

Stacked on the SQL-portability and joined-DML seam PRs (#1007, #1009); base branch is the joined-DML branch.

What

  • PostgresDialect.Rebind rewrites store-native ? placeholders to $n ordinals, leaving ? untouched inside single-quoted strings (including '' escapes), double-quoted identifiers, dollar-quoted bodies ($$…$$, $tag$…$tag$ — tags follow identifier rules, so a digit-leading $1 stays a parameter), -- line comments, and nested /* */ block comments.
  • PostgreSQL rendering for every dialect seam:
    • EXCLUDED.col upserts via ON CONFLICT … DO UPDATE, now() timestamps (microsecond precision is native), and interval arithmetic (now() - ? * interval '1 second').
    • InsertIfAbsent via ON CONFLICT (cols) DO NOTHING (conflict reports zero affected rows, same contract as INSERT IGNORE).
    • JSONBooleanIsTrue via jsonb_extract_path(...) IS NOT DISTINCT FROM 'true'::jsonb — null-safe like MySQL's <=>, so missing paths, JSON null, and SQL NULL yield false rather than NULL.
    • IndexHint renders empty — PostgreSQL has no index-hint syntax.
    • JoinedUpdate via UPDATE … SET … FROM … WHERE (join) AND (predicate); SET assignments stay textually ahead of the predicate so ?$n ordinal rebinding preserves the MySQL rendering's argument order.
  • Identity insertion via RETURNING idInsertID scans the generated key; InsertGuardedID maps the no-row case of a guarded INSERT … SELECT to inserted=false instead of an error.
  • NewWithDependencies now injects the configured identity/dialect into the task, apply-log, plan-comment, and settings stores, which previously pinned MySQLDialect{}. MySQL callers pass MySQLDialect{} for every seam; the one MySQL-visible change is that settings.Set now renews updated_at on same-value writes, so the column tracks the last write rather than the last value change — deliberate, documented in code, and covered by tests.
  • First parity lanes: the shared storagetest.TestSettings suite and an apply-logs round-trip run against a real PostgreSQL container, plus pgx stdlib value-contract pins (boolean, jsonb []byte, timestamptz round-trip precision, RowsAffected matched-rows semantics).

Why

The store core is dialect-parameterized but until now only had a MySQL implementation, so the seams were exercised on one side only. This nucleus makes the second dialect real and lets each remaining store family be brought onto PostgreSQL in small, individually verifiable slices.

Deliberately out of scope (follow-up slices): joined DML for the operation/task/apply stores, growing the PostgreSQL parity lanes to the remaining store families, and the public PostgreSQL store constructor.

Move INSERT IGNORE, JSON boolean predicates, and the index hint behind
the dialect; rewrite TIMESTAMPDIFF and SUBSTRING_INDEX portably. No
behavior change for MySQL.
Keep lease-guarded apply comment and control request updates portable
across dialect-specific joined DML syntax. No behavior change for MySQL.
PostgreSQL binder ($n rebind aware of strings, identifiers, dollar
quotes, and line comments), upsert/timestamp/interval rendering,
RETURNING-id identity insertion, and the first cross-dialect parity
lanes (settings, apply logs) plus pgx value-contract pins. Renders the
full dialect seam: ON CONFLICT DO NOTHING inserts, null-safe jsonb
boolean predicate, empty index hint, and UPDATE ... FROM joined
updates with SET placeholders ordered ahead of the predicate.
@Kiran01bm
Kiran01bm marked this pull request as ready for review August 12, 2026 06:32
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

PostgreSQL has no automatic timestamp renewal, so the settings upsert
now stamps updated_at explicitly and the heartbeat-stamp lint also
scans dialect JoinedUpdate call sites. Rebind no longer miscounts
ordinals around block comments or identifier-embedded dollar signs,
JSON path keys follow the plain-identifier panic contract, and
integration tests pin the lease-guarded RETURNING path and plain
timestamp round-trips.

@morgo morgo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Approving on Morgan's behalf (automated review, escalation rules apply).

Verified the risk areas: every ON CONFLICT target has an exactly matching unique index in pkg/schema/postgres (so PG's stricter named-target requirement can't throw); the ?$n rebinder correctly skips escaped strings, quoted identifiers, $$/tagged dollar quotes, and both comment styles in a single left-to-right pass so ordinals stay aligned with argument order; RETURNING id identity insertion mirrors MySQL's RowsAffected==0 guarded-insert semantics; de-pinning tasks/applyLogs/planComments/settings from the hardwired MySQLDialect is identity-preserving for MySQL; and the PG integration tests genuinely run in CI against postgres:16 (integration job passed, not skipped).

Non-blocking findings:

  • Nothing pins timezone=UTC on PG sessions — server-side now() cast into the schema's plain timestamp columns goes through session TimeZone, so lease-expiry/staleness predicates would skew on a non-UTC server, and the postgres:16 container's UTC default means this lane can't catch it. Suggest pinning timezone=UTC in the DSN in the public-constructor follow-up (#1012).
  • The PR body's "MySQL behavior is unchanged" slightly overclaims: settings.Set's upsert now stamps updated_at on a same-value write where the old ODKU left the row untouched. Deliberate, documented in-code, and covered — just worth a wording fix.
  • Rebind's dollar-quote scanner accepts digit-leading tags ($1$…$1$) that the PG lexer would treat as a parameter; unreachable with the store's ?-only corpus.

See the fencing note on #1011 — the PostgresDialect JoinedUpdate introduced here is the rendering in question.

Keep lease-guarded apply comment and control request updates portable
across dialect-specific joined DML syntax. No behavior change for MySQL.
The join condition's render position differs per dialect, so the
contract now forbids bind placeholders in it and defines the
assignments-then-predicate argument order. Empty assignment lists
panic at the seam instead of rendering invalid SQL.
@Kiran01bm
Kiran01bm force-pushed the kiran01bm/sqlstore-joined-dml-13k branch from 41ca96f to 3a25477 Compare August 13, 2026 00:05
…itions

A stray placeholder in the join condition would bind into a
dialect-dependent position and silently shift every subsequent
binding, so the seam panics on it instead of trusting the doc contract.
…k' into kiran01bm/pg-backend-nucleus

* origin/kiran01bm/sqlstore-joined-dml-13k:
  refactor(storage): reject bind placeholders in JoinedUpdate join conditions
  refactor(storage): tighten the JoinedUpdate dialect contract
  refactor(storage): render joined UPDATEs through the dialect
  feat(postgres): implement declarative planning via pg-sprite diffplan (#1008)
  refactor(storage): make remaining sqlstore SQL dialect-portable (#1007)
  test(e2e): deflake multi-table stop/start resume and MySQL cold starts (#1005)
  ci: verify golangci config against a vendored schema (#997)
  feat(api): fail-closed verdict gating for postgres plans (#1004)
  feat(tern): route postgres targets to the postgres engine (#1003)
  feat(storage): stamp remaining sqlstore timestamps explicitly (#1006)

# Conflicts:
#	pkg/storage/internal/sqlstore/apply_comments.go
#	pkg/storage/internal/sqlstore/dialect.go
#	pkg/storage/internal/sqlstore/dialect_test.go
#	pkg/storage/internal/sqlstore/settings.go
#	pkg/storage/internal/sqlstore/storage.go
#	pkg/storage/internal/sqlstore/updated_at_lint_test.go
Base automatically changed from kiran01bm/sqlstore-joined-dml-13k to main August 13, 2026 05:20
…nucleus

* origin/main:
  refactor(storage): render joined UPDATEs through the dialect (#1009)
  fix(planetscale): hold the cutover when the operator defers it (#978)
  fix(observability): make telemetry resource schema-tolerant (#1014)

# Conflicts:
#	pkg/storage/internal/sqlstore/dialect.go
#	pkg/storage/internal/sqlstore/dialect_test.go
#	pkg/storage/internal/sqlstore/storage.go
Copilot AI lite review requested due to automatic review settings August 13, 2026 05:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a PostgreSQL dialect “nucleus” to the shared pkg/storage/internal/sqlstore core, implementing PostgreSQL placeholder rebinding, SQL rendering seams, and identity insertion behavior so the same store logic can run against PostgreSQL, plus initial integration parity tests.

Changes:

  • Introduces PostgresDialect implementing rebinding (?$n) and dialect SQL seams (upserts, joined UPDATE, JSON boolean predicate, relative time, etc.).
  • Adds PostgreSQL identity insertion via RETURNING id, including guarded-insert semantics.
  • Adds first PostgreSQL integration parity lanes for Settings + ApplyLogs and pgx stdlib value contract tests; updates the updated_at stamping lint to account for dialect-rendered JoinedUpdate call sites.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
pkg/storage/internal/sqlstore/updated_at_lint_test.go Extends the source-scan lint to also validate updated_at stamping in dialect-rendered JoinedUpdate call sites.
pkg/storage/internal/sqlstore/storage.go Ensures all store families receive injected dialect/identity dependencies (removes hardwired MySQLDialect{} in a few stores).
pkg/storage/internal/sqlstore/settings.go Clarifies updated_at stamping expectations for upserts across dialects.
pkg/storage/internal/sqlstore/postgres_integration_test.go Adds PostgreSQL integration parity tests (Settings + ApplyLogs) and pgx stdlib value contract assertions.
pkg/storage/internal/sqlstore/identity.go Adds PostgreSQL identity insertion using RETURNING id, including guarded-insert no-row handling.
pkg/storage/internal/sqlstore/dialect.go Implements PostgresDialect (rebinding, upserts, joined UPDATE rendering, etc.).
pkg/storage/internal/sqlstore/dialect_test.go Adds unit tests covering PostgresDialect rebinding and SQL seam renderings.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread pkg/storage/internal/sqlstore/dialect.go
The PostgreSQL lexer reads a digit-leading run like $1 as a parameter
placeholder, never a dollar-quote delimiter, so the rebinder must not
skip placeholder rewriting between $1$…$1$ pairs. Unreachable with the
store's ?-only corpus; hardening for parity with server tokenization.
@Kiran01bm
Kiran01bm enabled auto-merge (squash) August 13, 2026 05:55
@Kiran01bm
Kiran01bm merged commit 3b637c4 into main Aug 13, 2026
33 checks passed
@Kiran01bm
Kiran01bm deleted the kiran01bm/pg-backend-nucleus branch August 13, 2026 06:04
Kiran01bm added a commit that referenced this pull request Aug 13, 2026
…-joined-dml-13l

* origin/main:
  fix(vitess): dispatch task-less VSchema-only work operations over gRPC (#961)
  feat(storage): add PostgreSQL dialect nucleus to the shared store core (#1010)

# Conflicts:
#	pkg/storage/internal/sqlstore/dialect.go
#	pkg/storage/internal/sqlstore/dialect_test.go
#	pkg/storage/internal/sqlstore/postgres_integration_test.go
#	pkg/storage/internal/sqlstore/updated_at_lint_test.go
@Kiran01bm

Copy link
Copy Markdown
Collaborator Author

Review response from Kiran's (@Kiran01bm) AI code review assessment agent (Amp / Claude Opus 4.5)

Summary: All three non-blocking findings are addressed — the dollar-quote tag fix landed on this branch, the UTC session pin is raised as a standalone PR, and the PR body wording is corrected; the fencing note will be answered on #1011.

# Finding Status Explanation
1 No timezone=UTC pinning on PG sessions — server-side now() into plain timestamp columns skews lease-expiry/staleness predicates on a non-UTC server fixed #1018 pins timezone=UTC in postgresconn's connection config (single choke point covering Open, OpenReloadable, and the credential-reload path) rather than the #1012 constructor, which takes an already-open *sql.DB. An explicit DSN timezone wins, mirroring sslmode. An integration test proves the session reports UTC even when the database default is overridden to a different timezone.
2 PR body's "MySQL behavior is unchanged" overclaims — settings.Set now stamps updated_at on same-value writes fixed PR body updated: it now calls out the one MySQL-visible change (updated_at tracks the last write rather than the last value change, deliberate and covered by tests).
3 Rebind's dollar-quote scanner accepts digit-leading tags ($1$…$1$) that the PG lexer treats as a parameter fixed 913bd72 — tag validation now follows identifier rules (no leading digit), with unit tests pinning both the digit-leading ($1$ stays a parameter) and underscore-leading ($_1$ still quotes) cases.
Fencing note on #1011 (JoinedUpdate rendering introduced here) deferred The fencing concern belongs to the #1011 review and needs a design decision (row locking vs. documented acceptance); it will be assessed and answered there.

Verified-correct section (ON CONFLICT targets, rebind ordinal alignment, RETURNING id semantics, identity-preserving de-pinning, CI lane genuinely running): no action — thanks for the confirmation.

Source: #1010 review 4919081588, posted by morgo (automated review on Morgan's behalf).

Kiran01bm added a commit that referenced this pull request Aug 13, 2026
…re-public-13n

* origin/main:
  refactor(storage): portable lease-guarded joined DML for the operation store (#1011)
  fix(github): align lint warnings formatting with issues and fold long lists (#959)
  fix(github): lead with the database's operators on command-rejection comments (#960)
  docs: regenerate stale tables of contents (#968)
  fix(engine): heartbeat the row a local drive actually owns (#915)
  fix(github): scope auto-plan to the schema a pull request proposes (#1016)
  fix(vitess): dispatch task-less VSchema-only work operations over gRPC (#961)
  feat(storage): add PostgreSQL dialect nucleus to the shared store core (#1010)
  refactor(storage): render joined UPDATEs through the dialect (#1009)
  fix(planetscale): hold the cutover when the operator defers it (#978)
  fix(observability): make telemetry resource schema-tolerant (#1014)
  feat(postgres): implement declarative planning via pg-sprite diffplan (#1008)
  refactor(storage): make remaining sqlstore SQL dialect-portable (#1007)
  test(e2e): deflake multi-table stop/start resume and MySQL cold starts (#1005)
  ci: verify golangci config against a vendored schema (#997)
  feat(api): fail-closed verdict gating for postgres plans (#1004)
  feat(tern): route postgres targets to the postgres engine (#1003)
  feat(storage): stamp remaining sqlstore timestamps explicitly (#1006)
Kiran01bm added a commit that referenced this pull request Aug 13, 2026
…lect-factory-14b

* origin/main:
  feat(github): flag destructive changes to tables another open PR owns (#1017)
  feat(storage): add public postgresstore constructor (#1012)
  refactor(storage): portable lease-guarded joined DML for the operation store (#1011)
  fix(github): align lint warnings formatting with issues and fold long lists (#959)
  fix(github): lead with the database's operators on command-rejection comments (#960)
  docs: regenerate stale tables of contents (#968)
  fix(engine): heartbeat the row a local drive actually owns (#915)
  fix(github): scope auto-plan to the schema a pull request proposes (#1016)
  fix(vitess): dispatch task-less VSchema-only work operations over gRPC (#961)
  feat(storage): add PostgreSQL dialect nucleus to the shared store core (#1010)
  refactor(storage): render joined UPDATEs through the dialect (#1009)
  fix(planetscale): hold the cutover when the operator defers it (#978)
  fix(observability): make telemetry resource schema-tolerant (#1014)
  feat(postgres): implement declarative planning via pg-sprite diffplan (#1008)
  refactor(storage): make remaining sqlstore SQL dialect-portable (#1007)
  test(e2e): deflake multi-table stop/start resume and MySQL cold starts (#1005)
  ci: verify golangci config against a vendored schema (#997)
  feat(api): fail-closed verdict gating for postgres plans (#1004)
  feat(tern): route postgres targets to the postgres engine (#1003)
  feat(storage): stamp remaining sqlstore timestamps explicitly (#1006)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants