perf(gen): JOIN a DISTINCT unnest(...) for composite-key relationship loaders & counts on PostgreSQL

[sandonemaki]

perf(gen): JOIN a DISTINCT unnest(...) for composite-key relationship loaders & counts on PostgreSQL

Builds on #714 (single-column col = ANY($1)). This PR optimizes the composite-key case.

Summary

Generated PostgreSQL composite-key relationship Slice loaders
(gen/templates/models/table/009_rel_query.go.tpl) and relationship counts
(gen/templates/counts/table/115_counts.go.tpl) previously filtered children with:

WHERE (a, b) IN (SELECT unnest($1::int8[]), unnest($2::int8[]))

The unnest subquery gives the planner a fixed row-count estimate, so it tends to
pick a Seq Scan + Hash Semi Join on the child table even when a usable index
exists.

This PR changes the composite-key filter to INNER JOIN a DISTINCT unnest(...)
of the parent keys:

INNER JOIN (
  SELECT DISTINCT k.a, k.b
  FROM unnest($1::int8[], $2::int8[]) AS k(a, b)
) AS keys ON child.a = keys.a AND child.b = keys.b

The planner can now use an Indexed Nested Loop (or combine per-column indexes
via BitmapAnd), while the parameter count stays bounded — one array per key column,
regardless of batch size.

Why JOIN (and not IN (VALUES ...) or a composite = ANY)

approach	params	plan	note
(a,b) IN (SELECT unnest(...)) (before)	bounded	mis-estimate → Seq Scan	slow on large child tables
(a,b) IN (VALUES (...), ...)	N × cols	good	parameter blow-up on large batches
JOIN unnest(...) AS k(a,b) (this PR)	bounded	Indexed Nested Loop	both benefits

Scope

• 009_rel_query.go.tpl — composite-key relationship Slice loaders.
• 115_counts.go.tpl — composite-key relationship counts.
• Single-column keys are unchanged (col = ANY($1) from perf(gen): use col = ANY(\$1) for single-column relationship loaders on psql #714).
• Non-PostgreSQL dialects are unchanged (row-value IN list).

Behavior (no functional change)

• Results are identical. DISTINCT prevents row multiplication from duplicate
  parent keys, so the loaded children and the counts match the previous output exactly.
• NULL semantics preserved — a NULL key matches nothing under both IN and the JOIN.
• Stitching / scanning untouched — composite-key stitching already used the
  nested-loop path, and the joined keys columns are not in the SELECT list
  (used only in ON), so the typed scan path is unaffected.

Runtime behavior: before vs after (by composite-index presence)

	Before (a,b) IN (SELECT unnest…)	After + composite index	After + no composite index
Generated SQL	row-value IN subquery	JOIN (DISTINCT unnest…)	same SQL as After + composite index (codegen doesn’t check indexes)
Typical plan	Seq Scan child + Hash Semi Join	Indexed Nested Loop	per-column indexes → BitmapAnd; none usable → Seq Scan + Hash Join
Child rows read	full scan once — O(M)	only matching rows via index	BitmapAnd: matching rows only; no usable index: full scan once — O(M), same as Before
Speed vs Before	—	faster than Before (no full scan)	per-column indexes → somewhat faster; no usable index → ≈ same as Before
Main bottleneck	large child table (full Seq Scan)	essentially none	large child table only when no index is usable — same as Before (full Seq Scan)

• M = child table rows, N = parents in the batch.
• “No composite index” ≠ always slow: per-column indexes can still yield a BitmapAnd. Only when no usable index exists at all does it fall back to Before’s Seq Scan.
• No regression: even the worst case (no usable index) matches Before — both Seq Scan the child once; the only added work is a DISTINCT over the bounded parent-key set.
• The speedup (vs Before) is guaranteed only when a composite index exists on the child's join columns.

Verification

• Generated against a PostgreSQL schema with a composite foreign key
  (references videos(user_id, sponsor_id)): the generated loaders
  (<Parent>Slice.<Relationship>) and counts (<Parent>Slice.LoadCount<Relationship>)
  emit the expected JOIN (SELECT DISTINCT ... FROM unnest(...) ...) and compile
  cleanly (go build ./models/).
• The SQLite codegen suite passes (template parses; non-psql output unchanged).

EXPLAIN ANALYZE benchmark (PostgreSQL 18.4, composite index on join columns)

Tested against a child table with a composite index on (par_a, par_b) and uniformly
distributed random data. The 50M-row cases use the exact SQL each template emits
(including type casts); smaller cases use equivalent SQL without explicit casts — the
plan is identical. The Returned rows column confirms both forms return identical results.

Rows	Batch	Before time	After time	Speedup	Returned rows	Est (Before)	Est (After)
100K	5	0.041 ms	0.016 ms	2.6×	1	2	1
100K	20	0.141 ms	0.062 ms	2.3×	4	40	2
1M	5	0.037 ms	0.016 ms	2.3×	6	25	5
1M	50	0.118 ms	0.085 ms	1.4×	42	2,500	50
10M	5	0.232 ms	0.062 ms	3.7×	45	250	50
10M	50	2.01 ms	0.27 ms	7.4×	470	25,000	500
10M	200	9.89 ms	2.09 ms	4.7×	1,932	400,007	2,000
50M	50	13.6 ms	0.19 ms	72×	96	5,002	100
50M	200	43.4 ms	0.98 ms	44×	406	80,032	400
50M	500	145.6 ms	2.35 ms	62×	1,042	500,202	400

Both forms use Index Scan on the composite index in all cases. The speedup comes from
row-count estimation accuracy: the old ProjectSet node (two separate unnest() calls)
over-estimates by 100–500×, causing buffer eviction and disk reads at scale. The new
Function Scan node (single unnest($1, $2)) estimates accurately, keeping all reads
in shared_buffers.

Checklist

• golangci-lint run reports 0 issues
• gofumpt -l reports no changes (changed files are .tpl only; flagged .go files are pre-existing and unrelated to this PR)
• Generated code compiles and codegen tests pass — PostgreSQL (pgx-v5-std: assemble / generate / generate_with_aliases)
• Generated code compiles and codegen tests pass — SQLite (modernc / ncruces: assemble / generate / generate_with_aliases)
• PostgreSQL EXPLAIN ANALYZE confirms composite index is used and results are identical before/after (see benchmark above)
• ./gen/... unit tests pass

[sandonemaki]

@stephenafamo
Hi, thanks for looking at this!

The failing check (Test / test (stable)) appears to be unrelated to the changes in this PR. I've confirmed that all tests pass locally.

[stephenafamo]

Can you rebase on main and try again?

[sandonemaki]

@stephenafamo

Can you rebase on main and try again?

Done — rebased on main. No conflicts.
Also added EXPLAIN ANALYZE benchmark results (PostgreSQL 18.4) to both #714 and #718.

[stephenafamo]

Kindly fix the merge conflicts, also make sure you're only including your commits, not commits from other PRs that have now been merged