Skip to content

feat(orm)!: preload child structs with a generated NULL-safe typed mapper - #725

Merged
stephenafamo merged 4 commits into
stephenafamo:mainfrom
sandonemaki:preload-typed-maper
Jul 7, 2026
Merged

feat(orm)!: preload child structs with a generated NULL-safe typed mapper#725
stephenafamo merged 4 commits into
stephenafamo:mainfrom
sandonemaki:preload-typed-maper

Conversation

@sandonemaki

Copy link
Copy Markdown
Contributor

Note: Builds on #715 (and benefits from stephenafamo/scan#8). Marked as draft until #715 lands — only the last commit is new.

Summary

This is the follow-up left "out of scope" in #715: Preloaded child columns were still scanned by
the prefix-aware, reflection-based scan.StructMapper inside the preload mapper mod. This PR
generates a second, NULL-tolerant typed mapper per table (<table>ScanMapperNullable) and passes
it to orm.Preload, so the joined child columns are also scanned without reflection.

Breaking change

orm.Preload (and the psql/mysql/sqlite Preload wrappers) gain a parameter to accept a
typed, NULL-safe mapper for the child struct. This lets the JOIN-based preload path skip the
reflection-based scan.StructMapper (+ NullTypeConverter + row validator) while preserving its
exact LEFT JOIN semantics: rows where every prefixed column is NULL still produce no child object,
and NULLs scanned into non-nullable fields still leave the zero value instead of erroring.

Who is affected: only hand-written callers of orm.Preload / <dialect>.Preload. Generated
code (the vast majority of users) is regenerated as part of this change and requires no manual
updates.

Migration: pass nil for the new parameter to keep the current reflection-based behavior
unchanged. Passing a generated <table>ScanMapperNullable opts into the typed path.

I considered an alternative, fully backward-compatible design (injecting the mapper via a new
PreloadOption instead of a signature change), but went with the explicit parameter for a simpler
buildPreloader implementation, and for symmetry with the constructor change in #715. Happy to
switch to the option-based approach if you'd prefer to avoid the breaking change — let me know and
I'll rework this PR.

Why the slice-loader mapper from #715 can't be reused as-is

The preload path needs three behaviours the fast mapper doesn't have (all currently provided at
runtime by StructMapper options in buildPreloader):

  1. Prefix matching — child columns arrive as <runtime alias>.<column> (the alias is
    generated per query, e.g. users_5.id).
  2. NULL tolerance — on a LEFT JOIN miss every child column is NULL, including columns whose
    struct fields can't hold NULL.
  3. Row validation — an all-NULL child row must produce no child object, not a zero-valued one.

What is generated

For every table that is the target of a to-one relationship (the only tables Preload can load —
this guard also keeps the unused linter happy):

  • a scan buffer struct whose fields use the nullable version of each column type (the same
    null types the models already use — no new wrapper types):
type videoPreloadBuf struct {
	ID        null.Val[int64]
	UserID    null.Val[int64]
	SponsorID null.Val[int64]
}
  • a mapper factory <table>ScanMapperNullable(prefix string) scan.Mapper[*Table] that resolves
    column names once per query (prefix check + name switch in the (ctx, cols) closure); the
    per-row path is one buffer allocation plus ScheduleScanByIndex calls. The after func returns
    nil, nil when every buffer field is invalid — the same contract as the current row validator
    (zero value, no error) — and otherwise copies the buffer into the struct, guarding non-nullable
    fields with a validity check so a NULL leaves the zero value exactly like the reflection mapper.

Because unselected columns are never scheduled and stay invalid, the static validity check over
all generated fields is equivalent to the runtime check over the selected subset, so
PreloadOnly/PreloadExcept behave identically.

Nested preloads compose unchanged: buildPreloader wraps the typed mapper with
scan.Mod(mapper(prefix), mapperMods...) instead of WithMapperMods (semantically equivalent).

Behaviour preservation

  • A new orm test (TestPreloadMapperParity) runs the same mock rows through the reflection
    mapper and a typed mapper and asserts identical results for: matched child, all-NULL child
    (LEFT JOIN miss → nil), partial NULL, and a selected-column subset.
  • End-to-end smoke on a generated SQLite schema (modernc driver): preload with matched/unmatched
    children, NULL values in nullable child columns, nested preload (posts → users → countries,
    where the middle table has a NULL FK), and PreloadOnly — results identical to the current
    behaviour, including runtime-generated alias prefixes.
  • One deliberate parity note: a child row that exists but whose selected columns are all NULL
    (e.g. PreloadOnly over nullable columns only) is dropped — same as today, since the current
    row validator also only sees the selected columns.

Performance

Preload scan-path micro-benchmark (BenchmarkPreloadMapper, committed), mock sql.Rows
scan.AllFromRows, parent (2 cols) + preloaded child (2 cols), -benchmem -count=3 (medians).
Numbers include the parent's scan cost, so the child-mapper delta itself is larger:

rows reflection typed Δ time allocs/op
100 71,035 ns/op 33,072 ns/op −53% 1,228 → 622
1,000 1,042,042 ns/op 508,335 ns/op −51% 12,031 → 6,025
10,000 11,348,437 ns/op 5,423,317 ns/op −52% 120,038 → 60,032

Testing

  • go test ./orm/... ./gen ./dialect/psql/... ./dialect/mysql/... ./dialect/sqlite/...: all pass.
  • go test ./gen/bobgen-sqlite/driver (generate → build → run generated test suites): modernc,
    ncruces and libsql pass. (mattn fails in my environment because CGO is unavailable — unrelated
    to this change.)
  • go test ./gen/bobgen-psql/driver (testcontainers): pass.
  • golangci-lint run on the touched packages: 0 issues; gofumpt -l: clean.

Out of scope

Generated <Parent>Slice.Load<Rel> loaders scanned related rows with
scan.StructMapper (per-row reflection). Generate a per-table reflection-light
<table>ScanMapper and use it instead, scheduling each result column directly
into the struct field by index. .All() merely delegates to bob.Allx with the
view scanner, so hooks, nested Preload/ThenLoad and behaviour are unchanged.
…pper

Generate a per-table NULL-tolerant typed mapper (<table>ScanMapperNullable)
for every table that is the target of a to-one relationship, and pass it to
orm.Preload so the joined child columns are scanned without reflection while
preserving the LEFT JOIN semantics of the previous scan.StructMapper path:
an all-NULL row still yields no child object, and NULL values scanned into
non-nullable fields still leave the zero value.

Builds on stephenafamo#715 and benefits from stephenafamo/scan#8.

BREAKING CHANGE: orm.Preload (and the psql/mysql/sqlite Preload wrappers)
gain a new PreloadMapper[T] parameter. Only hand-written callers of
orm.Preload are affected; generated code is regenerated as part of this
change. Pass nil to keep the previous reflection-based behaviour.
@stephenafamo
stephenafamo merged commit e423938 into stephenafamo:main Jul 7, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants