fix(db): align join keys with equality identity - #1834
Merged
Merged
Conversation
Canonicalize satisfiable join operands through the compilation ValueIdentity, keep nullish operands row-disjoint, and preserve raw lazy-demand values with stable representatives. This restores equality-predicate consistency for binary and other established value domains without adding compound join syntax. Provenance: #593 Lucas Duailibe and Sam Willis; #861 Vincent Chan; #896 Tomas Zaluckij; #779/#899/#1258 Kyle Mathews and Claude; #899 review by Sam Willis; #1229 Hieu Nguyen, Hiếu Nguyễn Minh, Kevin De Porre, and Claude Opus 4.6; #1797 ValueIdentity architecture by Kyle Mathews. Prior art informed the evidence and design; no prior implementation lines were reused. #593/#861 remain design-gated. Weight: production source net +30 lines; emitted join module +65 B ESM gzip and +60 B CJS gzip. No public API, export, dependency, compatibility branch, or db-ivm change.
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 8 included reviews per hour; 4 remain after this review. 📝 WalkthroughWalkthroughThe join compiler now preserves raw join values and uses predicate-compatible equality keys. Lazy loading skips nullish values and tracks raw demand values. Result processing no longer filters rows by join type. Tests cover equality, binary, temporal, opaque, nullish, lifecycle, and lazy-loading behavior. ChangesJoin Equality Alignment
Priority: ➖ Normal Estimated code review effort: 3 (Moderate) | ~25 minutes Change: Bug fix Sequence Diagram(s)sequenceDiagram
participant JoinPipeline
participant getJoinKey
participant LazyDemandTracking
participant CollectionLoader
JoinPipeline->>getJoinKey: derive key from raw join value
getJoinKey-->>LazyDemandTracking: provide generated join key
LazyDemandTracking->>CollectionLoader: issue IN-predicate load for non-nullish values
CollectionLoader-->>LazyDemandTracking: return candidate records
Merge Risk: ⚪ Minimal · up to The join equality alignment has focused regression coverage across supported value types and lazy loading. No concrete merge-blocking risk remains. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
More templates
@tanstack/angular-db
@tanstack/browser-db-sqlite-persistence
@tanstack/capacitor-db-sqlite-persistence
@tanstack/cloudflare-durable-objects-db-sqlite-persistence
@tanstack/db
@tanstack/db-ivm
@tanstack/db-sqlite-persistence-core
@tanstack/electric-db-collection
@tanstack/electron-db-sqlite-persistence
@tanstack/expo-db-sqlite-persistence
@tanstack/node-db-sqlite-persistence
@tanstack/offline-transactions
@tanstack/powersync-db-collection
@tanstack/query-db-collection
@tanstack/react-db
@tanstack/react-native-db-sqlite-persistence
@tanstack/react-router-with-db
@tanstack/rxdb-db-collection
@tanstack/solid-db
@tanstack/svelte-db
@tanstack/tauri-db-sqlite-persistence
@tanstack/trailbase-db-collection
@tanstack/vue-db
commit: |
Contributor
|
Size Change: -51 B (-0.03%) Total Size: 165 kB 📦 View Changed
ℹ️ View Unchanged
|
Contributor
|
Size Change: 0 B Total Size: 7.34 kB ℹ️ View Unchanged
|
This was referenced Sep 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Align live-query join keys with the query engine's existing equality identity. Joins now agree with
eqfor binary, Date, Temporal, scalar, BigInt/non-finite, and opaque-reference values; nullish operands are unmatched across direct, correlated, eager, and lazy joins.Root cause
The lazy-demand optimizer reused an already-normalized join key as the backend query operand. For binary values, the collection index then normalized that internal string as a user string, so double normalization prevented the candidate from matching. The graph needs the equality identity as its key while lazy demand retains the raw operand. Separately, full and correlated joins allowed nullish operands from opposite sides to share a bucket.
Approach
ValueIdentityexactly once.db-ivmalready guarantees the selected inner/left/right/full result shape, so the result mapper can be a direct pipe operator.db-ivm, public APIs, exports, dependencies, and compatibility branches are unchanged.Key invariants
eq, and a filtered cross join agree over every established equality domain.nullandundefinednever match in joins, including when row keys overlap across sides.db-ivmwithout a duplicate compiler filter node.Non-goals
and(eq(...), eq(...))join conditions from Allow joins with anand()condition #593/feat: support compound join conditions with and() (issue #593) #861 remain design-gated. The public join contract accepts one equality condition; this PR adds no join language or composite-key syntax.Trade-offs and package weight
Failing fast for established Date/Temporal/binary/reference equality would narrow existing query behavior, so the repair reuses the current identity relation instead. The emitted machinery is limited to a private
getJoinKeyhelper and one function-scoped demand-weight map; the map keeps one raw representative per live equality class while balancing inserts and retractions.JoinInputValueis type-only.The review pass replaced per-row nullish serialization with two inline side sentinels, removed an unused helper argument, and deleted a redundant post-join filter/closure. Recomputing the raw operand instead of retaining the tuple's third slot was rejected because it repeats expression evaluation and increased minified gzip by another 43 B ESM / 21 B CJS in the measured variant. Folding key policy into call sites and sharing unrelated demand bookkeeping were rejected because they reduce locality or broaden the change after the zero-growth target is met. Production TypeScript is net +1 line.
Against exact base
3ad64a42a0088e1272176fb33c953526fed9b868, every requested shipped-size measurement is now negative (85 modules per format, deterministic/usr/bin/gzip -n -9, Brotli q11/text):The complete normal/minified
disttrees shrink by 1,392 B / 766 B. The 641-entry npm package shrinks by 124 B packed / 1,616 B unpacked after a normal build and 78 B packed / 990 B unpacked after a minified build. There are no new exported state fields, public APIs, exports, dependencies, compatibility branches, package metadata changes, ordb-ivmproduction changes.Verification
3ad64a42a0088e1272176fb33c953526fed9b868: the 30-cell focused owner selection is RED (16 failed,14 path/control passes) across binary, Temporal, nullish, lifecycle, and raw lazy-demand paths.30 passed), with positive execution in bothautoIndex: offandeagermodes.4 failed), and an illegal upstream inner-join shape mutant is killed by five directdb-ivmowner tests.126/126passed.119/119; directdb-ivmjoin owners:60/60. Earlier equality/lifecycle collateral passed229/229; hash/value controls passed90/90; join/index boundary passed71/71.178files /5,930tests passed.36/36passed; the standalone invocation separately reports the repository's known 35 cross-packagerootDirdiagnostics. Normal and minified@tanstack/dbbuilds, including declaration generation, passed.git diff --check, and the commit hook passed.Key rerun commands:
Files changed
packages/db/src/query/compiler/joins.ts: unify join keying withValueIdentity, preserve raw lazy-demand values, and remove redundant result filtering.packages/db/tests/query/join.test.ts: permanent cross-formulation, lifecycle, disjointness, nullish, and lazy-work coverage..changeset/fix-query-join-equality.md: patch release note for@tanstack/db.Provenance and credit
eqequivalence exposed the binary inconsistency.inlineage; this PR retains raw demand values while using the current identity once.ValueIdentityrelation used here.Addresses the binary join inconsistency reported in #896 and credits/supersedes the approach in #899. No closure keywords are used because #896 is already closed, the other referenced records are pull requests or partial/design-gated work, and no material prior implementation code was copied (so no co-author trailers are warranted).