| doc | OVERVIEW | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| package | SubstrateML | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| repo | moot-core | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| authored_commit | f392a5ac680d0e934e7b93eb977ab1c00f2d333c | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| authored_date | 2026-07-23 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| sources |
|
The graph analytics calls now require explicit telemetry context.
CommunityDetection.detect, CommunityDetection.detectFull, and
EigenvalueCentrality.compute take estate and ts as required inputs.
A caller with no estate passes "".
A caller with no clock value passes 0.0.
This makes telemetry scope visible at every call site.
ConflictCue adds a deterministic pairwise conflict screen.
It detects changed values, one-sided negation, and revision markers.
Strong cues can support a reviewable contradiction proposal.
Borderline cues stay visible for higher-level judgment.
The Swift and Rust ports share the same tokenizer and score fixtures.
SubstrateML is the learning layer of the MOOTx01 substrate. MOOTx01 is an on-device AI memory system. It stores what an AI observes over time. It helps the AI recall that later. The substrate is the math base under that store. It holds the types, the bit operations, and the algorithms that every higher layer needs.
The substrate splits into layered packages. SubstrateTypes holds
pure data types. One example is the 256-bit row fingerprint.
SubstrateKernel holds hot-path bit operations. These run on every
capture. SubstrateML is layer three. It holds the cold-path
algorithms: math that learns structure from stored memories. It does
not store memories itself.
A memory in this system is a row. Each row has one fingerprint, one classification anchor, and a set of bitmap fields. SubstrateML never touches storage. Every function here takes plain values in and returns plain values out.
Most of these algorithms run during dreaming. Dreaming is the system's idle-time maintenance cycle. A background daemon runs it while the device sits quiet. The daemon clusters memories. It decays old evidence. It summarizes clusters and re-scores the memory estate.
An estate is one user's complete memory store. The algorithms here find the estate's themes, its communities, and its habits. They find its rules and its condensable clusters. They also forget on a schedule, so old evidence fades over time.
An on-device memory system must learn without a cloud. Cloud machine learning changes without notice. It needs a network. It sees private data. SubstrateML avoids all three problems. It ships small, exact, well-bounded reference algorithms. Every one of them runs entirely on the device.
The substrate must also learn the same way everywhere. MOOTx01 estates can federate: separate devices share and compare results. Two devices that factor the same matrix must get the same answer. Two devices that mine the same rules must also agree. Otherwise shared recall falls apart.
SubstrateML holds one agreement property across two implementations.
A Swift leg serves Apple platforms. A Rust leg, in rust/, serves
everything else. Both legs share one canonical pseudo-random number
generator, SplitMix64. Both use the same pinned seeds. Both use the
same tie-breaking rules and the same arithmetic order. The result is
bit-identical output on both legs. Conformance fixtures gate every
change. A fixture is a recorded input and output pair that both legs
must reproduce exactly.
Two library-wide rules protect that promise. First, SubstrateML never reads a clock. Every timestamp comes from the caller instead. Second, no algorithm here holds hidden state. Each one is a pure function or a plain value type. Each one is safe to run from any thread.
The package holds thirty-eight source files. They form seven working groups.
Fingerprint and distance math. A fingerprint is a short fixed-size
code computed from content. Similar content gives similar
fingerprints. FloatSimHash projects float embedding vectors from
external models into the substrate's 256-bit fingerprint form.
CompositeDistance blends classification distance and fingerprint
distance into one score. Recall ranks candidates by that score.
LatticeDistance, PartialStateRecall, and ShingleSimilarity supply
specialized distances for narrower needs. MomentSummary and
TemporalCompression reduce many row fingerprints into one signature
per time window. That lets the system compare "what was going on
during this hour" as a single lookup.
Ingestion shaping. FeatureExtractors turns raw ambient sensor
samples into fingerprinted rows. The samples come from health,
location, calendar, screen time, and telemetry sources.
AuditLogFold replays a row's append-only change log. That replay
reconstructs the row's state at any point in time. RowAttributeView
reshapes that same log into flat attribute lists. The pattern miners
consume those lists directly.
Learning and decay. MatrixDecay applies exponential half-life
decay to every statistics matrix. This is the system's forgetting
mechanism. ActionOutcomeMatrix tracks which actions succeed.
BradleyTerry learns a per-row ranking strength from recall feedback.
LLMCalibrationCurve tracks how honest a model's confidence claims
are. Sampling provides deterministic Normal, Gamma, and Beta
samplers. Thompson-sampling decisions draw on these samplers.
Graph analytics. The estate graph connects rows by association.
CommunityDetection runs Louvain clustering to find its communities.
EigenvalueCentrality scores each row's authority for keystone
recall. RandomWalks wanders the graph for exploratory recall.
NMFAlternatingLeastSquares factors the matrices into latent themes.
AnomalyDetection flags unusual values. JacobiSVD and FFT supply
deterministic linear algebra and rhythm analysis. Both sit beneath
semantic embeddings and periodicity detection. These five graph
algorithms emit telemetry signals when monitoring is enabled. The
signal names live in VizGraphSignals.
Pattern mining. AssociationRuleMining and AprioriMining find
rules of the form "when A is set, B tends to be set."
FormalConceptAnalysis and ConceptImplications find exact groupings
and implications that always hold. TemporalCausalityFold mines
statistics of the form "X changed, then Y changed some minutes later."
InformationTheory supplies the entropy and divergence math behind
these miners.
Distillation. Distillation compresses a cluster of related
memories into one condensed factoid. DistillationScorer decides
whether a cluster is coherent enough to compress.
DeltaFeatureExtractor and TypedDecayWeighting handle trends and
staleness in the underlying features. DistillationPipeline runs the
whole five-stage algorithm. It emits the factoid plus its
fingerprint.
Federation and privacy. PairingHandshake lets two estates derive
a shared fingerprint basis. It needs no extra network round trip.
TierContributionFingerprint packs an estate's shareable summary into
a fixed sixty-four-byte wire format. TierAscendingQuery and
DPORReduction add differential-privacy noise and k-anonymity. Both
protections keep any single estate's contribution hidden from
aggregate answers.
Figure 1 shows the library's topology. It shows the major parts and how data moves between them.
Figure 1. Topology of SubstrateML. Rows, audit events, and sensor samples enter on the left. The shaping layer feeds the four algorithm families. Dashed regions mark the substrate packages below and the telemetry seam. All outputs return to the calling kits, never to storage.
SubstrateML has no single facade. Each algorithm family is its own entry point. The consuming kits call the piece each one needs. These kits are LocusKit, CognitionKit, GeniusLocusKit, NeuronKit, and the dreaming daemon. A kit is a larger package that composes libraries into a subsystem. Kits depend on libs. Libs never depend back on kits.
The package depends downward only. It depends on SubstrateTypes for
shared types. It depends on SubstrateKernel for dispatched bit
kernels. It depends on IntellectusLib, the zero-dependency telemetry
leaf. Monitoring is off by default. When it is off, every telemetry
emit costs one atomic boolean load and nothing more.
The package ships the thirty-eight Swift sources. It ships a matching
test suite. It also ships the Rust port in rust/, crate
substrate-ml, with one module per Swift file plus shared conformance
tests. There are no bundled data artifacts. Every input comes from the
caller.
Determinism comes from pinned constants instead: seeds, thresholds, half-lives, and bucket tables. These live in the sources. Conformance vectors lock them in place. The same input therefore produces the same result on every platform, every time.