Skip to content

Commit e34da00

Browse files
mdhellermdheller
authored
Merge pull request #30 from SourceOS-Linux/feat/fog-layer-schema-skeleton
2 parents 4c0d3df + 00fc1e3 commit e34da00

18 files changed

Lines changed: 819 additions & 0 deletions

‎docs/FOG_ENVELOPE_CANONICALIZATION.md‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
# Fog Envelope Canonicalization and Signing Profile (v0)
2+
3+
This document defines the **recommended evidence profile** for FogVault and FogCompute envelopes until the rules are promoted into first-class schema constraints and ADRs.
4+
5+
## Goals
6+
7+
- deterministic hashing across heterogeneous nodes
8+
- small, auditable signing surface
9+
- stable content references independent of transport
10+
- explicit separation between content hashing, envelope hashing, and settlement logic
11+
12+
## Canonicalization profile
13+
14+
### 1. Canonical JSON
15+
16+
For any JSON payload that is hashed or signed, implementations should:
17+
18+
1. serialize with **JCS** (JSON Canonicalization Scheme; RFC 8785)
19+
2. encode as UTF-8 bytes
20+
3. hash with **SHA-256**
21+
22+
This yields the canonical `payloadHash`.
23+
24+
### 2. Content hashing
25+
26+
Raw content referenced by `ContentRef` should be hashed over the raw bytes, not over JSON wrappers. The digest form should be:
27+
28+
```text
29+
sha256:<lowercase-hex>
30+
```
31+
32+
### 3. Signature algorithm
33+
34+
The default signature algorithm for Fog envelopes and receipts should be:
35+
36+
- **Ed25519** for detached signatures
37+
38+
Recommended signature field encoding:
39+
40+
```text
41+
ed25519:<base64url-signature>
42+
```
43+
44+
### 4. Domain separation
45+
46+
To avoid cross-object signature confusion, signers should sign a domain-separated preimage:
47+
48+
```text
49+
SRCOS-FOG-V1\n
50+
<object-type>\n
51+
<object-id>\n
52+
<payload-hash>
53+
```
54+
55+
Where:
56+
- `<object-type>` is `TopicEnvelope` or `UsageReceipt`
57+
- `<object-id>` is the object URN
58+
- `<payload-hash>` is the canonical SHA-256 digest string
59+
60+
### 5. Encryption metadata
61+
62+
When envelopes are encrypted, the authenticated additional data (AAD) should bind at least:
63+
64+
- `topicId`
65+
- `entryId` or `receiptId`
66+
- `keyEpoch`
67+
- `payloadType`
68+
69+
Recommended default content cipher for topic payloads:
70+
71+
- **XChaCha20-Poly1305**
72+
73+
## Verification order
74+
75+
### TopicEnvelope
76+
77+
1. resolve and verify `ContentRef` digests if present
78+
2. canonicalize `payload`
79+
3. compute `payloadHash`
80+
4. verify detached signature
81+
5. only then evaluate topic semantics / policy
82+
83+
### UsageReceipt
84+
85+
1. canonicalize receipt body
86+
2. compute `receiptHash`
87+
3. verify detached signature
88+
4. verify referenced `WorkOrder` / output refs / settlement mapping
89+
90+
## What this does not settle yet
91+
92+
This v0 profile does **not** yet fix:
93+
94+
- key distribution / rotation mechanics
95+
- threshold or multisignature receipts
96+
- CBOR/COSE variants
97+
- transport-layer framing
98+
- receipt notarization / timestamp authority integration
99+
100+
Those should land in follow-up ADRs and schema revisions.

‎docs/FOG_LAYER.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Fog Layer (FogVault + FogCompute)
2+
3+
This document defines the **Fog Layer** contract surface for the SourceOS/SociOS ecosystem.
4+
5+
The fog layer is designed for **heterogeneous, churny, citizen-fog environments** where machines appear and disappear and cannot be treated as durable infrastructure.
6+
7+
## Architecture summary
8+
9+
### FogVault (storage + topic substrate)
10+
11+
- **Local substrate:** nodes expose fast, capacity-managed *local* storage (e.g., LVM-backed PVs via a local CSI driver such as TopoLVM).
12+
- **Distributed meaning + replication:** durability and convergence are provided *above* local storage via **append-only Merkle-log topics** (Hypercore). "Folders" and indexes are derived views.
13+
- **Topic contract:** a topic is an append-only channel with explicit replication policy, membership, and key epochs.
14+
15+
### FogCompute (compute substrate)
16+
17+
- Nodes advertise resource offers (CPU/GPU/RAM/IO windows).
18+
- Requestors submit work orders (image digests + inputs/outputs + verification policy).
19+
- Workers emit usage receipts (metered via OS primitives) and optional settlement events.
20+
- Settlement (tokenization / escrow) is a **pluggable layer**; receipts are the core evidence artifact.
21+
22+
## Contract status
23+
24+
The Fog Layer is represented by the schema set in `schemas/`:
25+
26+
- `Topic.json`, `TopicEnvelope.json`, `ReplicationPolicy.json`, `ContentRef.json`
27+
- `Offer.json`, `WorkOrder.json`, `UsageReceipt.json`, `SettlementEvent.json`
28+
29+
These are **v0 scaffolds** intended to be evolved via ADRs and semver discipline.

‎examples/content_ref.json‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"digest": "sha256:3d62e3b4c5dd6d04e3d1a2c765a809f0f76ad6f870d13dbb4d4ef7f77e4d2d10",
3+
"sizeBytes": 1048576,
4+
"mediaType": "application/vnd.srcos.fog.block+json",
5+
"uris": [
6+
"file:///srv/fog/topics/fog-cache-health/blocks/3d/62/e3b4",
7+
"hyper://fog-cache-health/blocks/sha256-3d62e3b4c5dd6d04e3d1a2c765a809f0f76ad6f870d13dbb4d4ef7f77e4d2d10"
8+
]
9+
}

‎examples/offer.json‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
{
2+
"id": "urn:srcos:offer:node-17-cpu-gpu-window-a",
3+
"type": "Offer",
4+
"specVersion": "2.0.0",
5+
"provider": {
6+
"subjectId": "urn:srcos:subject:fog-worker-node-17",
7+
"nodeId": "node-17"
8+
},
9+
"resources": {
10+
"cpuCores": 8,
11+
"memoryBytes": 34359738368,
12+
"gpu": {
13+
"count": 1,
14+
"model": "RTX-4090"
15+
},
16+
"storageBytes": 536870912000,
17+
"networkMbps": 1000
18+
},
19+
"availability": {
20+
"timeWindow": "2026-04-16T00:00:00Z/2026-04-16T08:00:00Z",
21+
"maxJobSeconds": 21600
22+
},
23+
"pricing": {
24+
"unit": "fog-credits",
25+
"cpuSecond": 0.002,
26+
"gpuSecond": 0.02,
27+
"gbHour": 0.005
28+
},
29+
"policy": {
30+
"allowWorkloadClasses": ["embedding", "indexing", "batch-inference"],
31+
"privacyMode": "topic-encrypted-only",
32+
"powerCapWatts": 350
33+
}
34+
}

‎examples/replication_policy.json‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"replicas": 3,
3+
"minAcks": 1,
4+
"neighborhoods": ["lan", "household", "region"],
5+
"retention": {
6+
"hotDays": 14,
7+
"coldDays": 90,
8+
"maxBytes": 53687091200
9+
},
10+
"compaction": {
11+
"mode": "snapshot",
12+
"intervalEntries": 50000
13+
}
14+
}

‎examples/settlement_event.json‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
{
2+
"id": "urn:srcos:settlement:index-qdrant-shard-20260416-01",
3+
"type": "SettlementEvent",
4+
"specVersion": "2.0.0",
5+
"receiptId": "urn:srcos:receipt:usage:index-qdrant-shard-20260416-01",
6+
"amount": {
7+
"unit": "fog-credits",
8+
"value": 12.84
9+
},
10+
"backend": "internal-ledger",
11+
"txRef": "ledger://fog-credits/2026/04/16/txn-000001"
12+
}

‎examples/topic.json‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
{
2+
"id": "urn:srcos:topic:fog-cache-health",
3+
"type": "Topic",
4+
"specVersion": "2.0.0",
5+
"name": "fog.cache.health",
6+
"description": "Health/cache topic for edge-local fog state with append-only Merkle-log replication.",
7+
"replication": {
8+
"replicas": 3,
9+
"minAcks": 1,
10+
"neighborhoods": ["lan", "region"],
11+
"retention": {
12+
"hotDays": 7,
13+
"coldDays": 30,
14+
"maxBytes": 10737418240
15+
},
16+
"compaction": {
17+
"mode": "checkpoint",
18+
"intervalEntries": 10000
19+
}
20+
},
21+
"encryption": {
22+
"cipherSuite": "xchacha20poly1305",
23+
"keyEpoch": 4,
24+
"keyRef": "urn:srcos:key:topic:fog-cache-health:epoch:4"
25+
},
26+
"members": [
27+
"urn:srcos:community:fog-workers",
28+
"urn:srcos:community:fog-observers"
29+
],
30+
"labels": ["fog", "cache", "ephemeral"]
31+
}

‎examples/topic_envelope.json‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
{
2+
"entryId": "urn:srcos:topic-entry:fog-cache-health:000001",
3+
"topicId": "urn:srcos:topic:fog-cache-health",
4+
"specVersion": "2.0.0",
5+
"occurredAt": "2026-04-16T02:15:00Z",
6+
"author": {
7+
"subjectId": "urn:srcos:subject:fog-worker-node-17",
8+
"nodeId": "node-17"
9+
},
10+
"payloadType": "PutBlob",
11+
"payload": {
12+
"path": "/cache/health/worker-17.json",
13+
"contentRef": {
14+
"digest": "sha256:3d62e3b4c5dd6d04e3d1a2c765a809f0f76ad6f870d13dbb4d4ef7f77e4d2d10",
15+
"sizeBytes": 1048576,
16+
"mediaType": "application/json"
17+
}
18+
},
19+
"integrity": {
20+
"content": {
21+
"digest": "sha256:3d62e3b4c5dd6d04e3d1a2c765a809f0f76ad6f870d13dbb4d4ef7f77e4d2d10",
22+
"sizeBytes": 1048576,
23+
"mediaType": "application/json"
24+
},
25+
"payloadHash": "sha256:9cd8d6d35b07be7db0d8904b5bd808c95bd1ef674353abce6ebf58c480644e12",
26+
"signature": "ed25519:example-signature-topic-entry"
27+
},
28+
"encryption": {
29+
"cipherSuite": "xchacha20poly1305",
30+
"keyEpoch": 4,
31+
"aad": "topic=fog.cache.health"
32+
}
33+
}

‎examples/usage_receipt.json‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
{
2+
"id": "urn:srcos:receipt:usage:index-qdrant-shard-20260416-01",
3+
"type": "UsageReceipt",
4+
"specVersion": "2.0.0",
5+
"workOrderId": "urn:srcos:workorder:index-qdrant-shard-20260416-01",
6+
"provider": {
7+
"subjectId": "urn:srcos:subject:fog-worker-node-17",
8+
"nodeId": "node-17"
9+
},
10+
"startedAt": "2026-04-16T02:20:00Z",
11+
"endedAt": "2026-04-16T02:28:42Z",
12+
"exit": {
13+
"code": 0,
14+
"status": "completed"
15+
},
16+
"usage": {
17+
"cpuSeconds": 3812.41,
18+
"memoryPeakBytes": 8912896000,
19+
"ioReadBytes": 721420288,
20+
"ioWriteBytes": 361758720,
21+
"gpuSeconds": 0
22+
},
23+
"outputs": [
24+
{
25+
"digest": "sha256:2222222222222222222222222222222222222222222222222222222222222222",
26+
"mediaType": "application/vnd.qdrant.snapshot",
27+
"sizeBytes": 285212672
28+
}
29+
],
30+
"integrity": {
31+
"receiptHash": "sha256:3333333333333333333333333333333333333333333333333333333333333333",
32+
"signature": "ed25519:example-signature-usage-receipt"
33+
}
34+
}

‎examples/workorder.json‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
{
2+
"id": "urn:srcos:workorder:index-qdrant-shard-20260416-01",
3+
"type": "WorkOrder",
4+
"specVersion": "2.0.0",
5+
"requestor": {
6+
"subjectId": "urn:srcos:subject:mesh-broker-1"
7+
},
8+
"workload": {
9+
"image": "ghcr.io/socioprophet/fog-indexer@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
10+
"command": ["/app/index", "--topic", "fog.cache.health", "--shard", "17"],
11+
"env": {
12+
"QDRANT_URL": "http://qdrant.service.local:6333",
13+
"TOPIC_ID": "urn:srcos:topic:fog-cache-health"
14+
}
15+
},
16+
"inputs": [
17+
{
18+
"digest": "sha256:1111111111111111111111111111111111111111111111111111111111111111",
19+
"sizeBytes": 52428800,
20+
"mediaType": "application/x-ndjson"
21+
}
22+
],
23+
"outputs": [
24+
{
25+
"digest": "sha256:2222222222222222222222222222222222222222222222222222222222222222",
26+
"mediaType": "application/vnd.qdrant.snapshot"
27+
}
28+
],
29+
"verification": {
30+
"mode": "replicate",
31+
"replicas": 2
32+
},
33+
"placement": {
34+
"requireGpu": false,
35+
"neighborhood": "lan"
36+
}
37+
}

0 commit comments

Comments
 (0)