Repository navigation
Diagram.restrict: apply Part conditions through the Master to preserve master-part compositional integrity #1501
Description
Summary
dj.Diagram.restrict is logically conjunctive (AND — it removes rows that fail the criteria).
Unlike the disjunctive trace/cascade (which include everything touched and keep/pull whole
compositional units), a conjunctive filter can cut into a master-part group, leaving partial
groups — a Master exported with only a subset of its Parts (or none). This preserves referential
integrity but breaks compositional integrity: the exported entity misrepresents the original.
Reproduction (structural)
dj.Diagram(schema).restrict(Master & 'm=1').restrict(Master.Part & 'p>3')Master→ all ofm=1.Master.Part→ (Parts ofm=1, from the master-keyed downstream restriction) ANDp>3→ only
thep>3Parts.- Result:
Master m=1whole, but only a subset of its Parts. A Master can even survive with zero
Parts, sincerestrictdoes not walk up from Part to Master.
Why it matters
- Referential integrity — fine. A kept Part's FK to its Master is satisfied; a Master needs no Parts
to be valid. The export re-imports without dangling foreign keys. - Compositional integrity — broken. DataJoint's master-part contract treats a Master's Parts as one
atomic entity. A recording exported with only some of its channels loads cleanly but is a
corrupted/partial entity.
Root cause / model
In DataJoint's model the Master is the entity; Parts have no independent existence. The unit of
export is therefore the Master-with-all-its-Parts. A Part-level filter is either meaningless for export,
or should mean "Masters that have a matching Part."
Decision (v2.4)
In Diagram restriction, conditions on Part tables are applied through their Master:
- A restriction that selects Part rows lifts to an existential restriction on the Master — Masters
that have a matching Part are selected. - A selected Master brings in all of its Parts.
- Net: compositional units are never split; Parts are never filtered within a unit.
This is the compositional-integrity analogue, for read-only restriction/export, of what cascade's
part_integrity="cascade" does for delete.
Scope / relationships
- Not for 2.3.1 — behavior change deferred to v2.4.
- Distinct from docs(diagram): document restrict is downstream-only (no part->master walk-up) #1500 (which established that
restrictdoes not performcascade's deletion
part→master walk-up). This issue is about filtering granularity — keeping units whole under a
conjunctive filter — a different mechanism; the two are compatible. - Optional 2.3.x stopgap: document the current behavior (restrict can yield partial Part groups; lift
Part criteria to the Master viaMaster & (Part & cond)).
References
- Surfaced in the 2.3.1
dj.Diagramreview (finding F12). - Code:
src/datajoint/diagram.py—restrict,_propagate_restrictions(mode="restrict"); contrast
_propagate_part_to_master(cascade's upward walk).
Activity
dimitri-yatsenko commentedon Jul 16, 2026
Scope note: this issue also subsumes documenting
restrict's contract (the intended docstring/spec text). PR #1500 attempted that against current 2.3.1 behavior ('downstream-only; never walks Part→Master') but was closed unmerged — that wording contradicts the direction here (Part conditions lifting to the Master). The restrict documentation should be written as part of this v2.4 work, against the final semantics: Part conditions apply through the Master; a matching Part selects its Master and brings in all its Parts; usetracefor upstream/ancestor closure.github-actions commentedon Aug 31, 2026
This issue is stale because it has been open for 45 days with no activity.