build: move the provisioning tail out of build and into provision

[mobileoverlord]

Owner: this PR is mine (the fast-rebuilds series), not the feeds series.
Both series push under the same account, so authorship does not distinguish
them.

Breaking change to what avocado build produces. Read the first section
before the diff.

What changes

avocado build and avocado runtime build now produce the deployable set:
the runtime manifest, the images and the elements. That is exactly what
avocado deploy and avocado connect upload read, and all either has ever
read.

The var image is a provisioning artifact — nothing but avocado provision
consumes it — so provision now builds it itself, at its entry point, ahead of
the avocado-provision-<arch> hook. Stone is untouched: it is invoked
exactly as before and handed exactly the inputs it has always been handed.

The OS bundle stays in build, along with the avocado-build-<target> hook
and the os_bundle manifest patch. See "Correction" below — an earlier revision
of this PR moved them too, and that broke OTA.

Migration

A pipeline that runs avocado build and then flashes will find no var image.
Run avocado provision, which produces it.

There is deliberately no flag to restore the old behaviour. A flag on
build would be asking the user a question whose answer is always "whatever the
next command needs" — and the flag design that was prototyped here made that
concrete: it needed a stamp for provision to interrogate, and an else-branch
that deleted both the artifacts and that stamp, because otherwise a plain
rebuild after a provisionable one left a stamp still matching unchanged inputs
while the artifacts it vouched for were gone. Moving the work removes the
question, the stamp and the failure mode together.

Why it was safe to move, checked rather than assumed

The section is ~200 lines, and its coupling to the build script turned out to be
small enough to relocate rather than reconstruct:

• Eight shell variables. Two (AVOCADO_PREFIX, AVOCADO_SDK_PREFIX) come
  from the entrypoint prologue that every container run already sources. The
  rest are deterministic paths, and the section now emits them itself, so it is
  self-contained.
• Six interpolated sections, all config-derived: Docker priming, mkfs flags,
  compression, losetup post-creation, device-tree overlays, LUKS room.

Nothing depended on build-time state that only the build script held. provision
already ran container commands and already read the var and stone config, so the
section slots in ahead of the existing hook call.

Shape

It moves to commands/runtime/var_image.rs behind one entry point taking an
explicit VarImageContext. That context is the point, not just tidiness: it is
precisely the set of things a portable provisioning bundle has to carry, so a
later avocado provision --bundle <path> becomes a matter of sourcing the
context from a bundle instead of from a project.

Tests

The behaviour moved with the code rather than being dropped:

• var-encrypt leaves room for the LUKS header, and a plaintext runtime keeps the
  tight image — now asserted against the rendered section.
• Device-tree overlays reach stone --overlay, and are inert when none are
  declared.
• The section defines every variable it reads that the prologue does not supply
  — the property a bundle depends on.
• bash -n on the rendered section, which is the check that catches a botched
  heredoc.
• build keeps a test that it emits none of the tail, anchored on
  start-of-line commands because "stone bundle" also appears in a comment that
  legitimately stays.

Addressed from Copilot review 1

All four findings were real; two of them were bugs that would have shipped.

• Wrong paths in the moved section. I wrote the preamble by hand and guessed:
  $AVOCADO_PREFIX/output/runtimes/<rt>/var-staging and an active symlink under
  lib/avocado/runtimes/. build actually writes $AVOCADO_PREFIX/runtimes/<rt>
  and links lib/avocado/active. It also read AVOCADO_NS_UUID and
  AVOCADO_IMAGES_DIR from the environment, which build used to export and no
  longer does. The section now resolves the manifest the way the hash collection
  does — the active symlink with a find fallback, because provision does not
  know the BUILD_ID that names the directory — and exports what the python step
  reads. the_section_uses_the_paths_build_actually_writes pins all of it,
  including that $AVOCADO_PREFIX/output appears nowhere.
• A leftover provisionable gate and flag. An earlier iteration of this work
  put a --provisionable flag on build and gated provision on a stamp. That
  design was dropped in favour of moving the work, but the flag, the
  StampCommand::Provisionable variant and the gate survived the rewrite. Since
  build no longer writes that stamp, the gate would have refused every
  provision. All of it is gone.
• Stone paths hard-coded alongside an existing OUTPUT_DIR. Now derived from it.
• Flag help text describing behaviour that no longer exists — removed with the flag.

Correction: the OS bundle is OTA payload, not a provisioning artifact

An earlier revision of this PR moved the whole tail to provision. That was
wrong and was caught on hardware (QCS9075, UKI platform), not by any test here:

• On UKI platforms the avocado-build-<target> hook is the kernel and
  initramfs — it assembles the per-slot uki_<slot>.efi that the stone update
  block names. With the hook only in provision, build emitted no boot
  artifacts at all.
• os-bundle.aos is the OTA payload, and stone bundle is also what populates
  os_artifacts. Without them the runtime manifest carried no OS payload.

Observed: avocado deploy reported success, updated the extensions, switched the
active runtime, and left the board on its old kernel with /var still
plaintext. Nothing reported that the OS had not been updated. That also made an
initramfs-only opt-in like var.encrypt unreachable over OTA, since its marker
lives inside the initramfs.

The rule now applied: anything an OTA requires is at the tail of runtime build; anything only provisioning consumes is at the start of provision.

Ordering turned out to matter as much as placement. The manifest id is minted by
manifest_section during build, so a bundle produced later by a separate
command gets patched into a document whose id has not changed — a device already
holding that id skips the update, including the os_bundle that appeared
inside it. Measured: a manifest genuinely carrying os_bundle still produced
"Runtime already at target version, nothing to do." Building the bundle in the
same run that mints the id removes that failure mode.

This also fixes sign_amf: command not found, which the moved code introduced:
the OTA half calls sign_amf and deliberately does not define it, inheriting the
helper from the build script. In provision that call resolved to nothing and
provisioning died after the bundle was built. A test now asserts the helper is
defined before the call in the rendered script, and that the re-sign follows the
patch it exists for.

The one inference I could not test on hardware

stone bundle needs a var partition size: every platform manifest declares var
as expand: "true" with no size, which is exactly what --partition-size
overrides, and stone fails hard without it — verified by running it:
partition 'var' omits size; no --partition-size override was supplied.

The var image does not exist at build time, so the size is now declared from the
staged tree plus headroom rather than measured from the image. I checked that
nothing acts on it: no provisioning script reads the bundle (avocado-provision-<target>
has zero references to it, and the UFS flow injects raw images), an OTA never
repartitions, and the partition expands at provision time. If a
provision-from-bundle path is ever added, that number becomes real and must come
from the image — which is written at the call site.

Split

commands/runtime/var_image.rs now exposes two halves over one context:
render_ota_tail (spliced into the build script, inherits its variables) and
render_var_image (standalone, emits its own preamble). Tests assert the split
in both directions, because getting it backwards is not a build failure — it is
a deploy that reports success and leaves the device on its old OS.

[Copilot]

🟡 Changes recommended

The moved provisioning tail currently has path/environment mismatches and provision still enforces a “provisionable build” gate with misleading messaging, which can cause provisioning failures or unnecessary rebuilds.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR relocates the “provisioning tail” (var image + stone OS bundle + Docker priming) out of runtime build and into runtime provision, introducing a dedicated renderer (commands/runtime/var_image.rs) and new stamp semantics to distinguish deployable build outputs from provisioning-only artifacts.

Changes:

• Add commands/runtime/var_image.rs to render/run the provisioning tail from provision (with unit tests around rendering and bash validity).
• Introduce a new Provisionable stamp command/requirement and wire it into runtime build/provision flows.
• Add a --provisionable CLI flag plumbing and document the breaking change in CHANGELOG.md.

File summaries

File	Description
src/utils/stamps.rs	Adds Provisionable stamp command plus constructors/requirements and tests.
src/main.rs	Adds --provisionable flags for build/runtime build and wires them into command builders.
src/commands/runtime/var_image.rs	New module rendering the provisioning tail script + tests.
src/commands/runtime/provision.rs	Runs the rendered provisioning tail before the provision hook; adds stamp gate helper/tests.
src/commands/runtime/mod.rs	Exposes the new var_image module.
src/commands/runtime/build.rs	Removes the provisioning tail from build output; optionally writes a provisionable stamp.
src/commands/build.rs	Plumbs provisionable through top-level build into runtime builds.
CHANGELOG.md	Documents the breaking change and migration guidance.

Review details

Suppressed comments (2)

src/commands/runtime/provision.rs:225

• This gate refuses provisioning unless a provisionable stamp exists, but this command now builds the var image and OS bundle itself later in execute() (via var_image::render), so the refusal can block an otherwise-valid avocado provision. The error message is also misleading because it claims the artifacts are built by avocado build --provisionable, but the artifacts are produced here in provision.

            // `avocado build` produces the deployable set only. The var image
            // and the OS bundle — the artifacts stone flashes — come from the
            // provisioning tail, which runs only under `--provisionable` and
            // records its own stamp. Require that stamp, and require it current:
            // a plain rebuild since then deleted those artifacts and the stamp

src/main.rs:1786

• Same issue as the top-level build command: this --provisionable help text says it builds provisioning artifacts, but the provisioning artifacts are now built during runtime provision. The CLI docs should reflect what the flag actually does (or the flag should be removed if it’s no longer needed).

        /// Also build the provisioning artifacts: the var image, the OS bundle
        /// and Docker priming.
        ///
        /// Without this, a build produces only what `avocado deploy` and
        /// `avocado connect upload` consume. `avocado provision` requires a build
        /// that used this flag.

• Files reviewed: 8/8 changed files
• Comments generated: 4
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[mobileoverlord]

@copilot review

[mobileoverlord]

All four findings from the review are addressed as of 9c8b901. Verified against the pushed head rather than the working tree:

Finding	State at 9c8b901
Wrong var-staging / active-symlink paths	VAR_DIR=$AVOCADO_PREFIX/runtimes/$RUNTIME_NAME/var-staging, manifest resolved via lib/avocado/active with a find fallback. $AVOCADO_PREFIX/output now appears exactly once in the file — in the assertion that it must not appear.
AVOCADO_NS_UUID / AVOCADO_IMAGES_DIR unset	Both exported by the section. build used to export them and no longer runs.
provisionable gate with misleading message	Gone. has_provisionable_build, the --provisionable flag and StampCommand::Provisionable were leftovers from a design that was dropped in favour of moving the work; since build no longer writes that stamp, the gate would have refused every provision.
Stone paths hard-coded beside OUTPUT_DIR	Derived from OUTPUT_DIR.

Two of those were bugs that would have shipped, and the gate one would have broken provisioning outright — thanks.

New test the_section_uses_the_paths_build_actually_writes pins the paths, the exported environment, and the symlink-then-search resolution, so this cannot regress silently. CI green.

Note for a re-review: the review summary above was re-timestamped by the force-push but its inline comments are outdated and its body is unchanged, so it reads as if nothing was addressed.

[Copilot]

🟡 Changes recommended

Two newly added doc comments are misattached to the runs_on option, producing misleading user-facing help/docs for remote execution.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 6/6 changed files
• Comments generated: 2
• Review effort level: Lite

[mobileoverlord]

Review round 2 addressed in 23355d6; all threads replied to and resolved. Verified against the pushed head, not a local tree:

#	Finding	State at 23355d6
1	var-staging / active-symlink paths, missing AVOCADO_NS_UUID + AVOCADO_IMAGES_DIR	fixed — $AVOCADO_PREFIX/output now appears once in the file, in the assertion that it must not appear
2	provisionable gate would refuse every provision	removed
3	stone paths hard-coded beside OUTPUT_DIR	derived from it
4	--provisionable help describing behaviour that does not exist	flag removed; main.rs is byte-identical to main
5	orphaned doc block adopted by runs_on in commands/build.rs	removed
6	same in commands/runtime/build.rs	removed

5 and 6 were caused by removing the provisionable field with a regex that matched the field but not its doc comment, so the block silently attached to the next field. That compiles and tests green — the only symptom is wrong user-facing help. I audited the rest of the commit for the same pattern and found none.

Gate: full cargo test (all 18 targets, not just --lib) and clippy --all-targets clean on this branch and on every branch stacked above it. CI green.

Two Copilot re-review requests have been accepted (201) since this push without a review arriving, so this comment is the record in the meantime.