Skip to content

Publish GB300 EKS evidence from a v1.0.0-rc1 run #2875

Description

@mchmarny

Summary

Publish GB300 EKS evidence from a v1.0.0-rc1 run, so the v1 tag ships with
evidence against the recipes v1 actually contains.

Split out of #2806. That issue asked for evidence at a released version and
is satisfied by the 0.21.1 run published in #2812 — "any release counts" was
the intended reading. This issue is the separate thing #2806's sequencing was
carrying implicitly: a run on the release candidate itself.

Why a released-version run is not enough

#2772 (GB300 node tuning, nodewright package bumps) merged 2026-09-18, nine
days after v0.21.1 shipped. The committed 0.21.1 pointers therefore certify
the untuned GB300 recipe. Every GB300 recipe change since is uncovered.

ROADMAP §2
requires GB300 training/Kubeflow and inference/Dynamo to "remain green against
the shipped recipes
". At the v1.0.0 tag the shipped recipe is the tuned one,
and nothing attests it.

This was called out on #2806 at the time: "Note v0.21.1 predates #2772 (GB300
node tuning), so these pointers certify the untuned recipe. Per the sequencing
in #2807 the definitive runs are on v1.0.0-rc1."
Closing #2806 on the
released-version reading is correct and leaves that observation untracked. This
is its tracker.

Scope

  • Run both coordinates against v1.0.0-rc1 and publish signed evidence
    • eks/gb300-ubuntu/training-kubeflow
    • eks/gb300-ubuntu/inference-dynamo
  • Confirm both Evidence links resolve against live board data
  • aicr evidence digest matches each pointer's predicate.recipe.digest
  • Publish without AICR_VALIDATOR_IMAGE_TAG set, so the attestation
    records an immutable validator tag (Validator image tag override silently replaces immutable provenance in evidence #2873)

Hardware

aws-us-east-2-nhensley-gb300, in the Sep 28-30 window. Per #2806: both GPU
nodes needed package repair (unattended-upgrades installed a 7.0.0 kernel EFA
3.0.0's DKMS cannot build against); the repair purged the linux-aws
metapackages, so the nodes no longer auto-advance kernels. Stable for a re-run;
the kernel-update policy remains an open question for the cluster owner.

If the window is missed

v1 ships with GB300 evidence at 0.21.1 against the untuned recipe. That is a
ROADMAP §2 carve-out, not a silent gap — amend the §2 acceptance text to say
so rather than leaving the criterion asserting something untrue.

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions