Tier-1 build-pipeline hardening: GCS Nix cache + self-managed signing/provenance#189
Merged
Conversation
magic-nix-cache (Determinate-sunset) threw 418/ResourceExhausted throttling on large builds — it disabled the substituter mid-run during the desktop build. Replace it everywhere with a persistent cache: - .github/actions/setup-nix: composite that installs Nix + (if NIX_CACHE_URL/ PUBKEY vars set) adds the GCS HTTPS prefix as a substituter. Graceful: with no vars it just falls back to cache.nixos.org (already more reliable than before). - scripts/nix-cache-push.sh: best-effort push of built closures to gs://<bucket>/nix-cache via signed file:// copy + gsutil rsync (no-op without NIX_CACHE_BUCKET/SECRET_KEY). build-custom-image.sh calls it after each build, so both the GitHub and GCP lanes seed the cache → later custom builds pull warm. - swapped all 5 workflows (release-images, build-custom, image-tests, nix-build-images, nix-ci) onto the composite; removed every magic-nix-cache use. - docs/NIX_CACHE_SETUP.md: one-time setup (keypair, public prefix, vars/secret). All YAML + scripts validate.
Every built image now gets a self-sovereign signature + provenance bundle — NO third-party signing authority (no Apple Developer ID, no Google/Play signing, no notarization). The key is an ed25519 minisign keypair we generate ourselves. scripts/sign-and-provenance.sh (best-effort, never fails the build): - minisign-signs each artifact with SOURCEOS_SIGN_SECRET_KEY (graceful no-op without it — still emits unsigned provenance). Falls back to `nix run nixpkgs#minisign` when minisign isn't on PATH. - emits an in-toto Statement (statementRef urn:srcos:attestation:…), a SLSA v1 provenance predicate (slsaPredicateRef urn:srcos:slsa:…), a Nix-closure SBOM, and for ISOs a sourceos-spec v2 OSImage doc whose provenance refs resolve to the above. OSImage validated against schemas.srcos.ai/v2/OSImage.json (ajv). - build-custom-image.sh calls it after the build and uploads the sidecars; the GCP lane passes the key (+version, +cache) via instance metadata. scripts/verify-image.sh: anyone-can-verify with the public key; also cross-checks the in-toto subject digest. Tampered artifacts are rejected (tested). build-custom.yml: SOURCEOS_SIGN_SECRET_KEY secret + SOURCEOS_SIGN_PUBKEY var + VERSION/SOURCE_REV. docs/SIGNING_SETUP.md: one-time keygen (minisign -W) + publish. Closes the gap where OSImage.provenance had no real attestation behind it.
mdheller
changed the title
Pre-existing failure on main: 57c9cab added `lib.mkForce` to hosts/exit-x86_64/default.nix's `sourceos.mesh.role = "exit"` to resolve a NixOS assertion conflict, but the contract grep matched the literal `role = "exit"` and so broke. Make the grep tolerate the optional mkForce wrapper — it validates the role value, not the exact syntax. Was blocking nix flake check on every PR.
Second pre-existing nix-ci failure, previously masked by the exit-contract failure: `shellcheck -S warning` flagged LIMA_KEY/IS_INTERNAL (deploy-stage2.sh) and EFI_SIZE (install-on-device.sh) as unused. These are intentional diagnostic probes in destructive disk scripts; annotate with shellcheck disable=SC2034 rather than touch the control flow. Verified clean with shellcheck 0.11.0.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two Tier-1 build-pipeline improvements that share
scripts/build-custom-image.sh.1. GCS-backed Nix binary cache (replaces sunset magic-nix-cache)
magic-nix-cacheis sunset by Determinate and threw418 / ResourceExhaustedthrottling — during the desktop build it produced ~1,571 disabled-substituter errors. Cold builds also rebuild every path..github/actions/setup-nix— composite: installs Nix + (whenNIX_CACHE_URL/NIX_CACHE_PUBKEYvars are set) adds the GCS HTTPS prefix as an extra substituter. Graceful — no vars → just Nix +cache.nixos.org.scripts/nix-cache-push.sh— best-effort push of built closures togs://<bucket>/nix-cache(signedfile://copy →gsutil rsync). Called bybuild-custom-image.sh, so both the GitHub and GCP lanes seed the cache → later custom builds pull the heavy edition base warm.magic-nix-cacheuse removed.docs/NIX_CACHE_SETUP.md.2. Self-managed signing + provenance (NO third-party authority)
No Apple Developer ID, no Google/Play signing, no notarization — a self-sovereign ed25519
minisignkeypair we generate ourselves.scripts/sign-and-provenance.sh— signs each artifact (graceful no-op without the key), emits an in-toto Statement, a SLSA v1 provenance predicate, a Nix-closure SBOM, and for ISOs a sourceos-spec v2OSImagedoc whoseprovenancerefs resolve to all of it. TheOSImageoutput is validated againstschemas.srcos.ai/v2/OSImage.jsonwith ajv.scripts/verify-image.sh— anyone-can-verify with the public key; cross-checks the in-toto subject digest. Tested: good → verified, tampered → rejected.build-custom.yml+ GCP startup script pass the key (+ version, + cache) through;docs/SIGNING_SETUP.mdcovers one-time keygen (minisign -W) and publishing the pubkey.One-time setup before either turns on: see
docs/NIX_CACHE_SETUP.mdanddocs/SIGNING_SETUP.md(generate keys, set the vars/secrets). Until then everything no-ops safely — nothing breaks.Validation: all YAML valid,
bash -nclean, sign+verify round-trip + schema-validation run locally.