Skip to content

Diagram notation: edge-weight rule is stated incorrectly; document the modernized style #246

Description

@dimitri-yatsenko

While restyling the LC-MS pipeline diagram for an upcoming blog essay, I worked through reference/specs/diagram.md § Visual Encoding and found the edge-style table states the weight rule incorrectly. Companion issue in datajoint-python covers the generator change; this one is the documentation.

The error

The current table reads:

Style Meaning
Thick line Master-Part relationship
Thin line Multi-valued foreign key

Both rows are wrong about what weight means. Line weight encodes cardinality, and it is binary:

  • Thick — the foreign key constitutes the child's entire primary key, so the dependency is 1:1: one parent row maps to at most one child row.
  • Thin — the child adds primary key attributes of its own, so the dependency is multi-valued: one parent row maps to many children.

Master-part is not a weight. A part table almost always adds a key attribute (scan_number, peak_idx), so under the actual rule its edge is thin. Attributing thick to master-part inverts the encoding on exactly the edges a reader is most likely to check.

Worked from datajoint/lcms-demo at main, where only three edges are genuinely 1:1:

Edge Child adds key attribute? Correct weight
Subject → Sample sample_id thin
Sample → Session session_idx thin
Session → Acquisition no thick
Acquisition → MassAnalysis no thick
MassAnalysis → PeakDetection key is the pair with PeakDetectionParams thin
PeakDetectionParams → PeakDetection same thin
Acquisition.Scan → MassAnalysis.Spectrum no — key is exactly Scan's thick
MassAnalysis.Spectrum → PeakDetection.Peak peak_idx thin
any master → its part scan_number / peak_idx thin

Note that the weight rule and the existing underline rule ("table introduces new primary key attributes") are two views of one fact: adding a key dimension is precisely what makes the incoming dependency multi-valued. Worth saying so in the doc — it helps a reader trust both cues.

Also worth documenting

Changes proposed alongside the generator restyle, so the spec and the renderer stay in step:

Tier colors, modernized. Same four hues, desaturated, each given a readable fill/stroke/text triple instead of alpha-blended primaries. Close enough that a familiar diagram stays familiar:

Tier Current Proposed fill / stroke / text
Manual green / darkgreen #E7F3EC / #2F7D5B / #1B5138
Lookup gray / black #F2F4F7 / #A9B1BD / #495261
Imported #00007F #E2ECFA / #2A5FA5 / #123A6D
Computed #FF0000 #FBEAEC / #B23A48 / #7C2430
Part transparent #FFFFFF / #9AA6B8 / #46536B

Shape stays load-bearing and unchanged: Manual rectangle, Imported and Computed ellipse, Lookup and Part plain.

Entity groups. A master and its parts enclosed together, using the same device as the schema grouping added in 2.1. The enclosure carries membership, so part labels can drop the master prefix and read as they do in the class body (Scan, not Acquisition.Scan), while staying smaller and muted to show they are subordinate. Where one master has several interdependent parts, the downstream part goes lower.

Direction. Left-to-right with no arrowheads — the layout carries direction, matching how the traditional-ERD comparison figures are drawn. Master and part share a rank so that horizontal reads as derivation and vertical as containment, which also keeps the figure compact.

Happy to open the PR against this page once the notation questions above are settled.

Activity

  1. added a commit that references this issue on Aug 13, 2026
  2. dimitri-yatsenko commented on Aug 19, 2026

    @dimitri-yatsenko
    MemberAuthor

    Adding a closing condition to this issue, prompted by @MilagrosMarin's review of #265.

    #265 brings § Visual Encoding's notation and theme hex triples in line with _DIAGRAM_THEMES as it stands after datajoint-python #1534/#1544. Both are merged but unreleased — the latest release, v2.3.2 (2026-07-21), predates them. So the spec table and the generated figures now document colors and edge styles the released library does not produce.

    Earlier PRs (#255, #256, #259, #263) already committed figures from the unreleased renderer, so this isn't new; what is new is that it now reaches spec depth, where a reader is entitled to treat the table as normative for the version they installed.

    This issue stays open until the renderer changes ship in a release, at which point:

    1. Re-verify § Visual Encoding's edge-style rows and theme hex triples against _DIAGRAM_THEMES in the released package.
    2. Regenerate the committed diagram SVGs against that release.

    Same shape as the once-per-release step #210 established for notebook version banners — one pass per major.minor, not per patch.

  3. dimitri-yatsenko commented on Aug 19, 2026

    @dimitri-yatsenko
    MemberAuthor

    Renderer side of the collapsed-edge issue is now open as datajoint/datajoint-python#1545.

    It changes what a collapsed figure looks like: an edge between two collapsed nodes no longer inherits the attributes of whichever foreign key in its bundle was visited first, and every bundle edge renders uniformly (solid, penwidth 2). An edge with only one collapsed end is unaffected and keeps its own cardinality and primary-vs-secondary styling.

    Consequence for the closing steps recorded above: when #1545 ships, pipeline-modules-collapsed.svg needs regenerating along with re-verifying the spec table — scripts/gen_pipeline_diagrams.py --check will flag it. Its four bundle edges become uniform, and lab → session in particular changes because its bundle mixes a primary and a secondary foreign key.

    Worth separating from the pydot padding caveat: padding entities are cosmetic, whereas the collapsed edge style was genuinely order-dependent, so that figure was not byte-reproducible across environments before #1545. After it, it is.

  4. dimitri-yatsenko commented on Aug 19, 2026

    @dimitri-yatsenko
    MemberAuthor

    Correcting my two comments above: I grafted a release-time re-verification obligation onto this issue and said it "stays open until the renderer changes ship." That was wrong on scope, and it would have held this issue open well past the work it actually asks for.

    This issue asks for two things. The edge-weight rule correction landed in #247 (merged Aug 13 — the commit referenced this issue without closing it, which is why it stayed open). The remainder — the tier palette, entity groups, and left-to-right/no-arrowhead direction, all deferred out of #247 so the spec and renderer stayed in step — is the diagram.md § Visual Encoding rewrite in #265. So this closes with #265, and #265 now says so.

    The re-verification obligation is a recurring maintenance step, not part of this ask, and it now lives in #266.

    One note for whoever reviews #265 against the table above: the hex triples proposed here (Manual #E7F3EC, and so on) are not what shipped. datajoint-python #1544 adopted the brand tier colors instead (Manual #e8f0e9 / #3e7a52), and #265 documents what the renderer actually produces rather than what this issue proposed.

  5. dimitri-yatsenko commented on Aug 19, 2026

    @dimitri-yatsenko
    MemberAuthor

    The brand guide governs the palette. The authority for diagram color is our brand guide — not the hex values proposed in this issue, and not "whatever the renderer emits." That guide ratified a different palette from the one above, so the table in the issue body is superseded.

    The substantive change: this issue proposed muted, invented tints, while the brand guide maps Imported and Computed onto the actual brand colors — DataJoint Blue and DataJoint Orange — under a "continuity first, closed set" principle whose stated test is naming every tier on an unlabeled diagram without hesitation.

    Tier Proposed here (fill / stroke / text) Brand guide, ratified
    Manual #E7F3EC / #2F7D5B / #1B5138 #E8F0E9 / #3E7A52 / #28513A
    Lookup #F2F4F7 / #A9B1BD / #495261 #F0F0F1 / #808285 / #5A5C5F
    Imported #E2ECFA / #2A5FA5 / #123A6D #E0F4FC / #00A0DF / #00537A — DataJoint Blue
    Computed #FBEAEC / #B23A48 / #7C2430 #FFEDE5 / #FF5113 / #B23200 — DataJoint Orange
    Part #FFFFFF / #9AA6B8 / #46536B #FFFFFF / #B9BBBE / #5A5C5F

    Also from the brand guide, and not in this issue: edges, cluster frames and titles are navy #171C39; renamed (aliased) foreign keys are bronze #C77D3A (dark #D68C4A), kept deliberately duller than DataJoint Orange so a renamed edge landing on a computed table doesn't read as the same signal. Shapes are unchanged and remain load-bearing, so the notation survives grayscale. The mark never appears inside a diagram (ruled 2026-08-17) — brand enters through color alone.

    datajoint-python #1544 implemented this palette, so _DIAGRAM_THEMES and the brand guide agree. The direction of truth is the brand guide → _DIAGRAM_THEMES → the docs spec table; #265 documents the shipped values because they are brand canon, not because the renderer is the authority. If the renderer ever diverges, the renderer is wrong.

    Two deliberate deviations to record, both documented in _DIAGRAM_THEMES: the schema-cluster frame is #171D3A rather than navy #171C39, and Part text is #55585C rather than Lookup's #5A5C5F. Neither is drift — the adaptive prefers-color-scheme block rewrites colors by exact value, so every light color must map to exactly one role, and the frame and the edge are both strokes needing distinct dark counterparts. Visually identical. The brand guide doesn't mention this constraint; worth adding there so a future audit doesn't file it as a defect.

    Status caveat: the brand guide is still marked 🟡 proposed — needs the brand owner's ratification and an upstream theme in datajoint-python, and its dark-mode values are marked owed while _DIAGRAM_THEMES already ships a full dark palette. So the implementation is ahead of the guide in one direction and the guide is authoritative in the other. Reconciling that belongs to the brand guide, not to this issue.

  6. dimitri-yatsenko commented on Aug 19, 2026

    @dimitri-yatsenko
    MemberAuthor

    Correcting the last two paragraphs of my comment above — I read a stale local checkout of the brand guide, and both claims are wrong.

    The brand guide is 🟢 shipped end to end, not 🟡 proposed: brand design ratified 2026-08-17, rendering mechanics in datajoint-python#1534, brand tier colors in #1544, docs restyle in #263. Dark-mode values are not owed — the guide has a full dark section, and it records that WCAG AA contrast and adaptive-mapping collision-freedom are under test in the suite.

    The guide also already documents both deviations I suggested adding to it, with the same rationale: Part text #55585C is called out as "one step off Lookup's, so every light value maps to exactly one role in the adaptive block," and the schema-cluster frame #171D3A as "one step off navy because edge and frame are both strokes and the adaptive block needs distinct dark counterparts." Nothing to add there.

    What stands from that comment: the brand guide governs the palette, the hex table in this issue's body is superseded, the direction of truth is the brand guide → _DIAGRAM_THEMES → the docs spec table, and #1544 implemented the ratified palette so the two agree. The apparent mismatch a reviewer might notice between this issue's proposal and #265's table is expected and correct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions