perf(gen): generate typed scan mappers and inject them via the model constructors - #715
Conversation
Generated <Parent>Slice.Load<Rel> loaders scanned related rows with scan.StructMapper (per-row reflection). Generate a per-table reflection-light <table>ScanMapper and use it instead, scheduling each result column directly into the struct field by index. .All() merely delegates to bob.Allx with the view scanner, so hooks, nested Preload/ThenLoad and behaviour are unchanged.
|
@stephenafamo |
|
@stephenafamo |
|
I like the direction, but I think the better fix is to pass the dedicated mapper to the table constructor and then use that to construct the queries. That way, all queries that are created for the models will flow through the typed mapper |
|
@stephenafamo While reworking this I also found a pre-existing bug ( One question on API shape: I added the mapper as a new parameter on the |
…pper Generate a per-table NULL-tolerant typed mapper (<table>ScanMapperNullable) for every table that is the target of a to-one relationship, and pass it to orm.Preload so the joined child columns are scanned without reflection while preserving the LEFT JOIN semantics of the previous scan.StructMapper path: an all-NULL row still yields no child object, and NULL values scanned into non-nullable fields still leave the zero value. Builds on stephenafamo#715 and benefits from stephenafamo/scan#8. BREAKING CHANGE: orm.Preload (and the psql/mysql/sqlite Preload wrappers) gain a new PreloadMapper[T] parameter. Only hand-written callers of orm.Preload are affected; generated code is regenerated as part of this change. Pass nil to keep the previous reflection-based behaviour.
… row The generated `<table>ScanMapper` ran a per-row `switch col` over every column to schedule scans. Resolve each column's index and field target once at query start (matching the child preload mapper in 105), so the per-row path only iterates the resolved targets and schedules by index.
…pper Generate a per-table NULL-tolerant typed mapper (<table>ScanMapperNullable) for every table that is the target of a to-one relationship, and pass it to orm.Preload so the joined child columns are scanned without reflection while preserving the LEFT JOIN semantics of the previous scan.StructMapper path: an all-NULL row still yields no child object, and NULL values scanned into non-nullable fields still leave the zero value. Builds on stephenafamo#715 and benefits from stephenafamo/scan#8. BREAKING CHANGE: orm.Preload (and the psql/mysql/sqlite Preload wrappers) gain a new PreloadMapper[T] parameter. Only hand-written callers of orm.Preload are affected; generated code is regenerated as part of this change. Pass nil to keep the previous reflection-based behaviour.
Summary
Generate a per-table, reflection-free
scan.Mapper[*T](<table>ScanMapper) and pass it to themodel constructor (
NewViewx/NewTablex), so that every query built from a model flowsthrough the typed mapper — as suggested in the review feedback.
This replaces the earlier approach of swapping the scanner inside the slice relationship loaders
only. The generated mapper scans each result column directly into the struct field by index
(a column-name
switch+ScheduleScanByIndex), avoiding the per-row reflection thatscan.StructMapperdoes (FieldByIndexper column plus an O(C²) column-name lookup per row).What flows through the typed mapper now
The
View.scannerfield is the single convergence point for model queries:View.Query(),Table.Insert/Update/Delete(RETURNINGscans) all copy it intoorm.Query.Scanner. Injectingthe mapper at construction switches all of them at once:
Model.Query().All()/One()o.LoadRel()os.LoadRel()(direct join).All(), no special-casing)scan.Mod(q.Scanner, …))Insert/Update/Delete+RETURNINGPreloadparent columnsPreloaded child columns (alias.col)The diff is small (+68/−36 across 14 files) precisely because the scanner really is a single
convergence point.
What changed
NewViewx/NewTablex(psql, mysql, sqlite) take ascan.Mapper[T]argument used for allqueries built from the view/table.
nilfalls back toscan.StructMapper, preserving the oldbehaviour.
NewView/NewTablekeep their current signatures (they passnil).xconstructors directly. If preferred,I can switch to a non-breaking form (e.g. a variadic option or a
WithScannermethod) — happyto adjust to whichever shape you want the API to have.
gen/templates/models/table/003_scan_mapper.go.tpl(moved from the loaders output):generates
<table>ScanMappernext to the model definition. It lives in the models outputbecause the constructor call references it and the loaders output can be disabled.
Since the constructor always uses the mapper, no unused mappers are emitted (this also resolves
the open question from the previous revision about a strict
unusedlinter).gen/templates/models/table/001_types.go.tpl(+ the mysql override): pass the mapper to theconstructor.
gen/templates/loaders/table/110_loaders.go.tpl: the loader-level special-casing isreverted. Direct-join slice loaders are back to plain
.All(ctx, exec); the through-join loaderwraps
q.Scannerinstead of naming the generated mapper, so the loaders templates no longerreference the mapper at all.
Safety / behaviour preservation
Correcting the safety claims from the previous revision (the earlier "unmatched columns are left
untouched" wording was imprecise):
destination fails at the
scan.Rowlevel (no destination for column X) regardless of whichmapper is used — neither mapper changes that. Columns consumed by another mod (preload
alias.colcolumns, the through-joinrelated_*columns) are scheduled by that mod, exactlyas before.
switchcases and themodel's
dbstruct tags are emitted from the same schema column names, and generated modelstag every column field explicitly (relationship fields are
db:"-"for both mappers).scan'sStructMappermatches column names with a plain==(no case folding or snake_case normalisation at match time — the snake_case default onlyapplies when deriving keys for untagged fields, which generated models don't have), the same
exact match the generated
switchperforms.Hooks,
mapperMods(nestedPreload) andloaders(nestedThenLoad) are applied insidebob.Allx/bob.Oneon top of the base mapper (scan.Mod(q.Scanner, mods...)), so they composewith the typed mapper unchanged.
Out of scope (possible follow-up)
Preloaded child columns are still scanned by the prefix-awareStructMapperinside the preloadmapper mod. Making those typed requires reproducing three runtime behaviours in generated code
(NULL-tolerant scanning for LEFT-JOIN misses, all-NULL row validation, and runtime-unique alias
prefixes), plus a small public API addition to
orm.PreloadSettings— I'd like to agree on thatAPI shape first, so it is left for a follow-up PR.
Performance
Scan-path micro-benchmark, mock
sql.Rows→scan.AllFromRows, generated 4-column table,Apple M4,
-benchmem -count=3(medians). Because the same injected mapper now serves every model query path, these numbers apply to the scan step of plain .All(), loaders and RETURNING scans alike (end-to-end gains depend on how much of each path is scan time):(Consistent with the −42…−48% measured in the previous revision on a 12-column table; the
real-workload numbers in the earlier description — eager-load −47…−60%, p95 −72% — measured the
same mapper swap on the slice-loader path and carry over.)
Testing
go test ./gen/bobgen-psql/driver ./gen/bobgen-mysql/driver ./gen/bobgen-sqlite/driver(testcontainers; generate → build → run generated test suites): all pass.
go test ./dialect/... ./orm/... ./gen: all pass.golangci-lint runon the touched packages:0 issues;
gofumpt -l: clean.plain
.All()(incl. nullable columns), one-object loader, direct-join and through-join sliceloaders (incl. empty relations),
INSERT … RETURNING, andPreload— verifying parent fieldsscan through the typed base mapper, the child relation is populated, and a parent row whose FK
is NULL (all child columns NULL) loads without error with the relation left
nil.