Proposal: Container Profile (corpus envelope) — spec §, example - #205
Open
perlowja wants to merge 4 commits into
Open
Proposal: Container Profile (corpus envelope) — spec §, example#205perlowja wants to merge 4 commits into
perlowja wants to merge 4 commits into
Conversation
Re-targets onto develop/v1.0.0 per @zircote (issue modeled-information-format#77, PR modeled-information-format#78) for the v1.0.0 cut. Editorial proposed section in SPECIFICATION.md before the Changelog, plus examples/container-profile.corpus.json and a CHANGELOG [Unreleased] entry. Reference converters ship separately (issue modeled-information-format#77).
…-information-format#77) The Container Profile section claimed reference converters (Mem0/Letta/ Cognee/Graphiti/MemPalace) 'are provided separately'. They exist today as MPF-era converters in the CHARON tooling and do not yet emit this MIF Container Profile envelope, so the claim overstated the current state. Reworded to 'in progress (modeled-information-format#77)' pending the MIF-native port.
2 tasks
zircote
added a commit
that referenced
this pull request
Jul 11, 2026
…1) (#261) * adr: propose Container Profile transport envelope (ADR-021) Settles the corpus-envelope design the maintainer separated out of issue #77 / discussion #60 into its own ADR: an OPTIONAL, single-file *.corpus.json transport envelope derived from a Bundle, built entirely from already-merged per-unit decisions (PR #83 conceptType, PR #84/ADR-014 DocumentReference, PR #85 provenance shape), with vendor-specific corpus metadata routed through a generalized extensions mechanism rather than becoming native MIF vocabulary. Links added bidirectionally in the related ADRs' frontmatter per the adr/README.md convention. * feat(schema): Container Profile envelope schema, context, and validation (ADR-021) Implements ADR-021's Decision Outcome: schema/container.schema.json (the MemoryCorpus envelope, kind-discriminated records[] union, corpus-level provenance/editChain/extensions), schema/container-context.jsonld (its JSON-LD context), schema/document-reference.schema.json (a standalone $ref wrapper onto mif.schema.json's DocumentReference $defs entry, since ajv-cli cannot take a #/$defs/... fragment appended to -s directly), scripts/validate_container.py (the dedicated CI-visible validator *.corpus.json files need, since they're invisible to the .md-only okf_validate.py/mif_convert.py gates by construction), a new container-validation job in .github/workflows/validate.yml, and a fully verified worked example under examples/container/. Two defects found and fixed during implementation, beyond the original design draft: extensions is mapped @type:@JSON rather than @container:@index, since the latter was confirmed (via pyld expand) to silently drop its own value content on JSON-LD expansion -- the exact class of defect this whole design exists to fix, recurring in a new field. kind values (memory/document) are now registered vocabulary terms so they expand to real mif: IRIs instead of garbage relative ones. Verified end-to-end: both new schemas compile; the worked example validates against container.schema.json; every record's payload validates against the correct external schema per kind; the full envelope round-trips losslessly through pyld expand/compact (records[] and extensions both survive, unlike PR #205's design); the existing okf_validate.py, mif_convert.py roundtrip, schema-validation ajv loop, and Astro build are all confirmed unaffected by these additive changes. * docs: Container Profile reference doc and spec cross-reference (ADR-021) Adds docs/CONTAINER-PROFILE.md (envelope shape, kind-discriminated records[] rules, the extensions mechanism and its promotion rule, local validation instructions, Bundle round-trip notes), a one-sentence cross-reference from both spec surfaces (SPECIFICATION.md and its Starlight mirror overview.mdx, kept in sync per this repo's existing dual-surface convention), and a CHANGELOG entry under Unreleased. Updates ADR-021's Audit section with a second, dated entry recording the real implementation evidence (file:line citations against the actual schema/script/CI/example/doc files added across the two prior commits) and verification performed (validate_container.py PASS, both schemas compile, existing gates unaffected, pyld round-trip lossless, Astro build clean). Status stays Proposed and this entry is marked Partial, not Compliant: item 5 of Decision Outcome (@perlowja's confirmation on issue #77) is an external, social step this local work does not and should not attempt. * fix: address local review findings (ADR-021) A high-effort local /code-review pass (9 independent finder angles) on the full three-commit diff surfaced real defects, all fixed here: - schema/container.schema.json's provenance.wasDerivedFrom was a hand-rolled, narrower reimplementation (object-only, @id-only, additionalProperties:false) that would reject valid PROV shapes mif.schema.json's own ProvNode explicitly permits (bare strings, id as well as @id, arrays) -- directly contradicting ADR-021 Decision point 4's claim to reuse the per-unit shape "exactly". Now a real $ref onto mif.schema.json's ProvNode $def. - scripts/validate_container.py's envelope-validation call was missing the extra_refs plumbing the ProvNode $ref now requires, which broke validation of the repo's own shipped example (caught by re-running the validator after the schema fix, before this ever reached CI). - validate_container.py's main() reported "PASS (0 file(s) checked)" and exited 0 for an empty or nonexistent directory -- the identical silent-pass failure mode this script exists to prevent (ADR-021 Decision point 9). Now fails closed on zero files found. - Malformed JSON in a *.corpus.json file crashed with an uncaught traceback instead of a clean per-file error. - The default directory ("examples/container") was CWD-relative, silently checking nothing when run from any directory other than the repo root despite the module docstring advertising it as a supported invocation. Now anchored to REPO_ROOT. - The new container-validation CI job had no Setup Python step, unlike every sibling Python-invoking job in the same workflow file, which all pin python-version 3.12. - ADR-021's Audit table cited schema/container.schema.json lines 133-217 for the Record $defs block; the file is 172 lines and the real block is 108-170. Corrected, along with the CI-job line range (shifted by the Setup Python addition) and the compile-verification command (missing --spec=draft2020). - Documented, rather than silently left implicit: the extensions losslessness claim is verified manually (pyld is not a CI dependency), and two pre-existing, out-of-scope defects in the core schema/context.jsonld (the per-unit extensions field's identical @container:@index content-loss bug, and documentType's missing @vocab fallback) surfaced by this work but not fixed here, since fixing either means editing code every memory unit in production already depends on. Re-verified end-to-end after every fix: validate_container.py PASS, both schemas compile, existing okf_validate.py/mif_convert.py roundtrip/Astro build all unaffected, pyld expand/compact still lossless on the corrected files. * docs(adr-021): correct stale defect-tracking note now that #224/#225 are fixed The Neutral-consequences note describing two pre-existing context.jsonld defects (per-unit extensions @container:@index content-loss, documentType missing @vocab fallback) as open follow-on work is now stale: both were independently filed (#224, #225) and fixed/closed the same day by unrelated vocab-term-scoping work already merged into main and picked up by this branch's sync. Updated the ADR text and stale line-number citations to reflect the resolved state instead of describing live defects. * fix(container-profile): address local /code-review findings A high-effort local /code-review pass (8 finder angles, 1-vote verify) on the full branch diff surfaced real defects, fixed here: - schema/container-context.jsonld's editChain entries mapped their `@id` and `supersedes` fields with `@type: "@id"`, so opaque transport-lineage identifiers (e.g. "container-transfer-042") were silently rewritten to base-relative IRIs on JSON-LD expand -- the same content-mutation-on- expand defect class this ADR fixed elsewhere (documentType, extensions), unaddressed here. Renamed the field `@id` -> `transferId` (schema + context + description) so it's an ordinary term, not a JSON-LD node identifier, and dropped `@type: "@id"` from `supersedes` -- both now expand as plain string literals, verified via pyld. - schema/container.schema.json's `@context` property allowed any URI string, so a corpus using the bare per-unit context (https://mif-spec.dev/schema/context.jsonld, which ADR-021 Decision point 8 explicitly forbids at the envelope level) validated as fully `valid`. Added a `const`/`contains` constraint requiring the container context URL for the string/array forms; verified a tampered instance using the wrong context now fails ajv validation. - scripts/validate_container.py's validate_corpus() assumed every `records[]` entry was a dict; a non-object entry (e.g. a bare string) crashed with an uncaught AttributeError, losing any already-collected envelope-schema errors. Added an isinstance check that reports a clean per-record error instead; verified against a constructed bad-shape corpus. - scripts/snapshot-schema-version.py's MIRRORED_FILES list (drives the versioned schema mirror generated before each release) never gained the three new Container Profile files, so they'd have silently missed public/schema/<version>/, /latest/, /vMAJOR/, and index.json despite container.schema.json declaring a canonical, advertised $id. Added all three. - .github/workflows/schema-check.yml's independent "Mirror alias consistency" job has its own hardcoded file list (a separate mechanism from MIRRORED_FILES) with the identical gap. Added the same three files there too. Re-verified end-to-end: validate_container.py PASS on the real example, FAIL (clean, no traceback) on both constructed bad-shape corpora, pyld expand no longer mutates editChain identifiers, ajv rejects a wrong-@context envelope. * docs(adr-021): convert Audit table citations to durable anchors The 2026-07-10 (implementation) Audit table cited raw line numbers (e.g. ".github/workflows/validate.yml, 97-124"), one of which had already gone stale after the branch's merge-with-main shifted the container-validation job down by 9 lines (caught in local code review). Rather than just correct the number, convert every row to the durable-anchor convention (job id / step name / heading text / field name / "full file") this workspace already retrofitted 11 other ADRs to use (#238, #241, #245-249), since a fresh ADR authored after that convention was established should follow it from the start rather than reintroduce the same stale-citation failure mode. Verified clean against scripts/check_adr_audit_citations.py. * refactor: extract shared ajv-cli invocation into scripts/_ajv_common.py scripts/validate_container.py's _ajv_validate and scripts/validate-ontologies.py's _ajv_validate were two independent, already-diverging reimplementations of the same subprocess-invoke-ajv-and-capture-errors logic (npx vs bare ajv, missing 20-line truncation and --strict=false in the newer one, differing input contracts) -- a future ajv-invocation fix landing in one would not propagate to the other. Extracted the shared subprocess/tempfile/error-handling logic into scripts/_ajv_common.py; each caller now wraps it with its own thin prefix/truncation/extra-refs behavior. Also brings validate_container.py's invocation up to --strict=false, matching the sibling script's established choice and quieting ajv's strict-mode compile advisories that were otherwise getting mixed into real validation-error output. Re-verified: validate_container.py PASS on the real example; validate-ontologies.py and test_subtype_of.py both still produce identical output against the test/subtype_of fixtures (7/7 subtests pass). * docs(adr-021): link the two remaining out-of-scope findings to filed issues The local /code-review pass surfaced two real, genuine-design-tradeoff findings that don't have a mechanical fix within this branch's scope: the missing pyld/CI regression check for container-context.jsonld's extensions mapping, and the two context files hand-copying shared prefixes/terms instead of composing. Both were prose-only in the ADR text (a "fix it now or file it" gap for the first one specifically); filed as #257 and #259 respectively and linked from the ADR instead of left as unfiled narration. Also replaced one remaining raw schema/context.jsonld line-number citation in Decision point 8's prose with a durable description, consistent with the Audit table's conversion. * fix(container-profile): batch validate_container.py's ajv calls by kind (#260) _ajv_common.py's ajv_validate spawned one npx/ajv subprocess per record, paying full npx resolution + schema (re)compilation for every record even though same-kind records share a schema. Add ajv_validate_batch, which issues one ajv-cli invocation per kind-group (repeated -d flags) instead of one per record, cutting an N-record corpus down to (distinct kinds) + 1 spawns. Per-record error attribution verified against a mixed valid/invalid batch to confirm ajv-cli's per-file result ordering holds. Adds scripts/test_ajv_batch.py as a permanent regression check (mixed-batch attribution + subprocess spawn count) and wires it into the container-validation CI job. * fix(container-profile): pyld CI regression check for container-context.jsonld (#257) ADR-021's own Consequences/Negative section already documented this gap: nothing in CI performed a real JSON-LD expand/compact against schema/container-context.jsonld, so a future edit reintroducing @container:@index for extensions (the exact content-loss defect this ADR exists to prevent) would silently regress with no error signal anywhere. Add scripts/test_container_context_fidelity.py, the container-profile counterpart to test_jsonld_context_fidelity.py: expands/compacts extensions through real pyld and asserts it survives as an opaque @JSON literal. Confirmed the check actually catches the regression by reverting the term locally and observing the predicted symptom (nested keys silently dropped). Reuses the already-hash-pinned requirements-jsonld-ci.txt (no new dependency), installed into the container-validation CI job. Updates ADR-021's Consequences/Negative section to record this as fixed rather than deferred. * fix(container-profile): CI drift-check between container-context.jsonld and context.jsonld (#259) container-context.jsonld hand-copies content that already exists verbatim in context.jsonld (the mif/prov/xsd prefixes, extensions, wasDerivedFrom) instead of composing the two -- and this already diverged once in this branch's own history until issue #224 reconciled it. Nothing forced the two files to stay in sync. Resolved via #259's option (c): add scripts/check_container_context_drift.py, a structural CI check that fails if the shared prefix/term definitions diverge between the two files, without restructuring either one. Composing via an array @context (option a) was rejected: it would import context.jsonld's unrelated per-unit vocabulary into the container envelope's root active context, against ADR-021 Decision point 8's "minimal blast radius" driver. Confirmed the check catches real drift by deliberately mismatching a prefix locally and observing the expected failure. Wired into the container-validation CI job. Updates ADR-021 Decision point 8's note to record this as resolved. * docs(adr-021): add 2026-07-11 follow-up-fixes Audit entry (#257, #259, #260) Consolidates the three follow-up fixes landed on this branch (ajv batching, container-context.jsonld pyld regression check, shared-term drift check) into one dated Audit entry, per this repo's re-audit-after-implementation convention. Still Partial, not Compliant: item 5 (@perlowja's confirmation on issue #77) remains outstanding. * docs: reconcile ADR-002/ADR-005/SPECIFICATION.md stale ADR-003 prose (#251) SPECIFICATION.md §2.1 "Dual Representation" still opened with the original ADR-002 co-equal framing ("MIF defines two equivalent representations"), never reconciled with ADR-011's canonical/derived refinement even though the Abstract and §6's heading in the same document were. Reworded §2.1 to name Markdown canonical and JSON-LD derived, matching both. ADR-002's own Related Decisions bullet for ADR-003 was factually wrong: it described Obsidian compatibility as a live design constraint, but ADR-017 superseded ADR-003 and dropped that commitment. Updated the bullet, added a new one for ADR-017, and added ADR-017 to ADR-002's related: frontmatter (ADR-017's own related: list already pointed back at ADR-002 -- the cross-reference adr/README.md requires was one-way). ADR-005 had the same superseded-ADR-003 problem independently, in its Context and Decision sections. Both now state the underlying fact directly (the underscore prefix is an ordinary visible character in every common file browser/editor, Obsidian included) with a pointer to ADR-017, instead of citing ADR-003 as a live guarantee. Adds a 2026-07-11 (follow-up) Audit entry to both ADR-002 and ADR-005 recording the fix, per this repo's re-audit-after-implementation convention. * docs: enumerate SPECIFICATION.md's Invariant N references in one place (#252) SPECIFICATION.md, several ADRs, MIGRATION.md, and docs/okf-conformance.md all cite specific "Invariant N" numbers in prose, but no enumerated list defining what Invariants 1-6 (or however many exist) collectively are existed anywhere -- confirmed via repo-wide grep. ADR-009's own 2026-07-11 audit already surfaced and filed this gap as #252 without fixing it, since the fix is documentation content, not a mechanical citation-format change. Add a "### Invariants" section to SPECIFICATION.md, right after the Abstract's positioning table, enumerating Invariants 2-6 with their source citations. No Invariant 1 or 7+ is asserted: no existing citation anywhere in the repo names one, and the Abstract's "OKF-compliant... (a tested invariant)" bullet -- the closest candidate for a future Invariant 1 -- is noted as a candidate, not asserted as one, since no citation currently numbers it. Point the scattered inline references (SPECIFICATION.md's own Abstract/ §2.1/§6, MIGRATION.md, docs/okf-conformance.md, ADR-009) at the new section via anchor links instead of leaving them as bare unlinked numbers. Adds a 2026-07-11 (follow-up) Audit entry to ADR-009 recording this as fixed. * fix: address local 8-angle review findings on the #257/#259/#260 fixes An 8-angle diff review of the prior three commits (ajv batching, container context fidelity, drift check) surfaced real correctness bugs before pushing: - validate_container.py's grouping crashed with an uncaught TypeError on a malformed, unhashable `kind` value (a list/dict), where the old `kind == "memory"` comparison chain never crashed on any input. Simplified the generic _KIND_SCHEMAS/dict-of-dicts grouping back to two explicit memory/document blocks -- fixes the crash and removes an abstraction layer that added indirection for exactly two hardcoded, closed-set kinds. - ajv_validate_batch's dict-vs-Path branching in _ajv_common.py treated any non-dict JSON value (a string/list/number/null payload) as a filesystem path, crashing on Path()/read_text() instead of letting ajv report a real schema-validation error. Now only actual Path instances are read from disk; everything else is JSON-serialized directly. - ajv_validate_batch attributed batched results to instances purely by positional order of ajv-cli's output blocks. Now parses the instance name back out of each `<path> valid/invalid` result line (each temp file is already named `<name>.json`), so attribution is unambiguous by construction instead of depending on an undocumented ordering guarantee. Also surfaces any output preceding the first result line to every instance instead of silently dropping it, and fails closed if a result line can't be matched to a known instance. - Error output is emitted in original records[] document order again (previously grouped by kind, since kind-groups were validated separately). - ajv_validate is now a thin wrapper over ajv_validate_batch instead of a ~30-line near-duplicate of its subprocess/cmd-building logic -- the module's own docstring already states single-copy-of-this-logic as its purpose. - test_ajv_batch.py's scratch schema now uses tempfile instead of writing into scripts/ directly. - check_container_context_drift.py's CHECKS list computes its comparison pairs eagerly at import time instead of deferring through a lambda (the values were already fully loaded before CHECKS was built). - Added the missing **Audited revision:** commit-SHA line to the four new 2026-07-11 (follow-up) Audit entries in ADR-002/005/009, per adr/README.md's "record the audited revision for each dated Audit entry" requirement -- present in every pre-existing entry, missing from these. All fixes verified by direct reproduction before and after: the unhashable- kind crash, the non-dict-payload crash, and the document-order regression were each empirically confirmed against the pre-fix code and confirmed fixed against the post-fix code. Full local gate suite re-run clean. * fix: make test_container_context_fidelity.py hermetic, no live network fetch CI failed on push: "pyld.jsonld.JsonLdError: ('No default document loader configured',)" against https://mif-spec.dev/schema/context.jsonld. container-context.jsonld's `payload` term points its own scoped @context at that remote URL, and JSON-LD 1.1 context processing resolves every term-scoped @context reachable from an expanded @type's active context up front -- not lazily only for properties actually present in the document. So even a test document with no `payload` key still triggered a real network fetch attempt on every expand() call; this passed locally only because a document loader happened to be available there, while CI's environment (only cachetools/frozendict/lxml/pyld pinned) has none configured by default. Add an explicit, hermetic documentLoader passed via options to every expand()/compact() call in this test: resolves the one known URL to the real local schema/context.jsonld file, and raises for anything else instead of ever touching the network. Confirmed the fix by forcing the global pyld document loader to always fail (reproducing CI's exact failure condition) and re-running the test in a fresh process -- passes clean. * fix: address xhigh /code-review findings on the Container Profile branch Correctness (all pyld/repro-verified before fixing): - container-context.jsonld: a bare kind:document DocumentReference payload silently lost its hash object on JSON-LD expand (core context defines hash only inside the documents term's scope) -- payload's scoped context now carries the documents-scoped hash definition; and the provenance scoped context dropped id-keyed ProvNodes and every PROV relation other than wasDerivedFrom -- it now registers the id alias plus the same PROV relation terms context.jsonld defines top-level. Both hand-copies are drift-checked and regression-tested. - _ajv_common.ajv_validate_batch fail-opened: an instance ajv exited nonzero without reporting (killed mid-batch) came back with an empty, valid-looking error list; now fails closed per instance. stdout/stderr are parsed as separate streams (ajv-cli prints valid to stdout, invalid + details to stderr; a merged pipe can interleave under backpressure and mis-attribute errors). Also: UTF-8 pinned, npx anchored to the repo root, path-separator instance names rejected, truncate=0 no longer means uncapped. - validate_container.py: a non-object *.corpus.json crashed the whole run with AttributeError; a typo'd directory argument was silently ignored whenever another argument matched. Both now fail closed. CI scans examples/ docs/ profiles/ instead of only examples/container. Serving/release pipeline: - public/schema/index.json now catalogs the three new schemas (canonical URL only) so the next release snapshot publishes them; RELEASING.md 1c documents the schema/ -> public/schema/ sync the snapshot script requires; docs/CONTAINER-PROFILE.md and ADR-021 note the canonical URLs resolve at the next schema release, not at merge. - schema-check.yml compiles container.schema.json and document-reference.schema.json (with the mif.schema.json ref bundle); previously neither was compile-checked by the workflow whose stated purpose is compiling every schema document. Docs/bookkeeping: - ADR-021: Audited revision SHAs added to all three Audit entries (the adr/README.md requirement this branch enforced on ADR-002/005/009 but missed here), frontmatter updated: bumped, stale 'SPECIFICATION.md line 97' citation converted to a durable anchor, Decision point 8 documents the scoped-term copies. - CONTAINER-PROFILE.md no longer claims the per-unit extensions term uses @container:@index (fixed to @type:@JSON by #224). - CHANGELOG 'Added (proposed)' folded into the standard Added category; README/examples index reconcile the examples-layout prose with examples/container/. - check_container_context_drift.py compares the full shared-term intersection dynamically (documented allowed divergence for provenance) and loads inside main() so malformed input is a clean FAIL, not a traceback. - test_container_context_fidelity.py: schema-invalid '1.0' fixture version corrected, context URL imported from mif_convert, new document payload hash and ProvNode-form round-trip regression checks. - validate.yml: npm/pip caching in the container-validation job. - RELEASING.md picked up MIF L1 frontmatter (id/type/created) to satisfy the mif-docs fail-closed guard on edit. * docs(adr-021): accept ADR-021, merged with its implementation Maintainer decision 2026-07-11: acceptance rides the implementation merge rather than remaining gated on issue #77 confirmation (which stays welcome as post-acceptance feedback). Status flipped proposed -> accepted in the ADR frontmatter, Status section, Decision Outcome, and a new dated acceptance Audit entry (audited revision 11d92a3); every downstream 'proposed' status reference updated to match (adr/README.md index, docs/CONTAINER-PROFILE.md header, CHANGELOG entry, SPECIFICATION.md and overview.mdx cross-references, README/examples index notes). * fix(ci): install requirements-ci.txt in container-validation job test_container_context_fidelity.py now imports mif_convert (for CONTEXT_URL), which needs pyyaml -- the job only installed the jsonld requirements, so the fidelity step died with 'PyYAML required' in CI while passing locally where pyyaml was already present. * docs(spec): drop the time-stamped absence claim from the Invariants note Copilot review: the repo-wide 'no citation names Invariant 1 or 7+' claim was already contradicted by this PR's own audit prose and rots the moment a new invariant is numbered. The note now describes the enumerated set (2-6), why the OKF-compliance bullet stays unnumbered, and how the list grows.
Signed-off-by: Robert Allen <zircote@gmail.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR proposes adding a “Container Profile (Corpus Envelope)” section to the MIF specification, alongside a worked *.corpus.json example and a changelog entry, aiming to describe a transport envelope for moving multiple memory units plus source-document references as a single artifact.
Changes:
- Adds a new “Container Profile (Corpus Envelope) — PROPOSED” section to
SPECIFICATION.md. - Adds a new example corpus file:
examples/container-profile.corpus.json. - Adds an “Added (proposed)” changelog entry describing the Container Profile.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 13 comments.
| File | Description |
|---|---|
| SPECIFICATION.md | Introduces a proposed Container Profile section (envelope shape, record kinds, corpus-level fields, conformance). |
| examples/container-profile.corpus.json | Adds a worked JSON example of a MemoryCorpus envelope. |
| CHANGELOG.md | Adds an Unreleased “Added (proposed)” entry for the Container Profile. |
Comment on lines
+2241
to
+2245
| ## Container Profile (Corpus Envelope) — PROPOSED | ||
|
|
||
| > **Status:** Proposed addition (see issue #77, "Adopt the MPF Corpus Envelope | ||
| > as a MIF Container Profile"). Placement is editorial — slotted here for the | ||
| > v1.0.0 review; promote/renumber into the body as fits the cut. |
Comment on lines
+2259
to
+2262
| "@context": "https://mif-spec.dev/schema/context.jsonld", | ||
| "@type": "MemoryCorpus", | ||
| "mif_version": "0.1.0", | ||
| "records": [ ], |
Comment on lines
+2263
to
+2264
| "provenance": { }, | ||
| "edit_chain": [ ] |
Comment on lines
+2275
to
+2279
| | `document` | a source document (e.g. an unmodified DoclingDocument) the memories derive from, so provenance resolves within the corpus | | ||
|
|
||
| `fact` and `event` are **not** separate kinds — they are MIF memories whose | ||
| proposed `memoryCategory` term is `Fact` or `Event`. This avoids a parallel | ||
| schema. |
Comment on lines
+2285
to
+2288
| | `provenance` | W3C-PROV provenance for the corpus as a whole | | ||
| | `edit_chain` / record `supersedes` | supersession / version history across records | | ||
| | `compression_manifest` | (Level 3) which records are compressed summaries of which sources | | ||
| | `federation_cursor` | cursor for incremental / federated corpus transfer | |
Comment on lines
+21
to
+22
| "memoryType": "semantic", | ||
| "namespace": "_semantic/requirements", |
| }, | ||
| { | ||
| "kind": "memory", | ||
| "memoryCategory": "Fact", |
Comment on lines
+34
to
+35
| "memoryType": "semantic", | ||
| "namespace": "_semantic/requirements", |
| } | ||
| } | ||
| ], | ||
| "provenance": { "@type": "prov:Entity", "prov:wasDerivedFrom": "ncp-requirements.pdf" } |
Comment on lines
+10
to
+17
| ### Added (proposed) | ||
|
|
||
| - **[Container Profile]**: corpus envelope (`*.corpus.json`) wrapping many MIF | ||
| memory units + source `document` records for transport (issue #77). Editorial | ||
| spec section + `examples/container-profile.corpus.json`. Reference converters | ||
| (Mem0, Letta, Cognee, Graphiti, MemPalace) that emit this envelope are in | ||
| progress (#77). PROPOSED — pending | ||
| v1.0.0 incorporation. |
zircote
requested changes
Jul 15, 2026
zircote
left a comment
zircote
left a comment
Collaborator
There was a problem hiding this comment.
Thank you for putting this proposal together. The envelope needs a few adjustments before it will validate against container.schema.json and mif.schema.json:
- Envelope: mif_version is not a recognized field here; use containerProfileVersion instead. mif_version already means something different (an implementation's declared spec-conformance version in .mif/config.yaml), so reusing the name for the envelope's own version was rejected as a naming collision (ADR-021, decision point 5). @context also needs to point at https://mif-spec.dev/schema/container-context.jsonld, not the bare memory-unit context.
- records[2]: remove memoryCategory. It is not an allowed record field; this was already considered and rejected (see PR #83).
- records[0] (the document): the embedded DoclingDocument payload does not match what kind: document expects. It needs to be a DocumentReference pointer (@type: "DocumentReference", plus url or id) rather than the full document dump. See ADR-014 and ADR-021 for the rationale: documents are referenced, not embedded.
Separately: I found a real gap in the schema while reviewing this. memoryType was documented as a backward-compatible alias for conceptType, but the schema never actually accepted it as sufficient. That is issue #274, now fixed and merged. Your memory-kind records validate as-is with memoryType; no change needed there.
…schema - Drop the proposed CHANGELOG entry (Container Profile shipped in 1.3.0 via a separate implementation; this entry duplicated the real one). - examples/container-profile.corpus.json: use containerProfileVersion (not mif_version) and the container-context @context per schema/container.schema.json; the document record is now a DocumentReference pointer instead of an embedded DoclingDocument dump; drop the disallowed memoryCategory record field (see PR modeled-information-format#83); use conceptType (schema-required) instead of memoryType; fix the provenance object to use the plain wasDerivedFrom key the schema defines, not the prov:-prefixed variant. Validated: python3 scripts/validate_container.py examples/ -> PASS (2 files).
Author
|
Fixed in 11ec5db: dropped the now-duplicate "Added (proposed)" CHANGELOG entry (Container Profile already shipped in 1.3.0), and |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft spec PR for issue #77 — adds a Container Profile section: a transport envelope around MIF memory units, so a whole corpus (many memories + the source documents they derive from) moves as one artifact.
records[]withkind: memory (a MIF memory unit, validated, levels apply) | document (e.g. an unmodified DoclingDocument).fact/eventfold into MIF memories via a proposedmemoryCategoryterm — no parallel schema.provenance(W3C-PROV),edit_chain/supersedes,compression_manifest,federation_cursor.examples/container-profile.corpus.json+ CHANGELOG.Draft — section number is editorial, and this is yours to shape/merge. The reference converters (mem0, letta, cognee, graphiti, mempalace) + docling import + validator are staged at perlowja/mif-converters (still happy to transfer into the org the moment public-repo creation is enabled for members, per the #77 thread). Pairs with #77.
Rebased 3-file diff onto
main:SPECIFICATION.md(+ Container Profile §),examples/container-profile.corpus.json,CHANGELOG.md. Example JSON validated well-formed. Standing by on thememoryCategoryplacement decision to align spec + converters.