Skip to content

Latest commit

 

History

History
249 lines (207 loc) · 14.3 KB

ci-validation.md

File metadata and controls

249 lines (207 loc) · 14.3 KB

Native platform evidence

Rust core: Dagger and Actions

.github/workflows/native-core.yml runs the Linux checks through the local Dagger module in .dagger/. Native Windows and macOS runners execute the same tools/check_native.sh script directly. Dagger's Linux container does not establish Windows or macOS correctness. The Go SDK is CI tooling only; the product core remains Rust.

From the repository root on Windows, with the Podman machine running and the native Dagger CLI on PATH (PowerShell):

dagger call evidence --source . --revision (git rev-parse HEAD) export --path D:\BuildArtifacts\lesscode\native-evidence
dagger call check --source . --revision (git rev-parse HEAD)

No WSL distribution is involved; the CLI starts its engine as a container in Podman. On 2026-11-10 check passed this way in 250 s from a cold engine with CLI 0.21.11, while dagger.json and the Actions workflow stay pinned to 0.21.8. On Linux or macOS use the same two commands with "$(git rev-parse HEAD)" and a local export path.

Use local-working-tree as the revision for uncommitted changes. The revision argument is a provenance label, not an integrity check on the supplied directory. The Actions checkout and its recorded SHA identify the tested CI revision. No host Go installation or Dagger Cloud account is required. Module bindings are generated by Dagger and excluded from Git; commit the module source and Go locks.

The pipeline pins Rust 1.98.1, nextest 0.9.143 and Dagger 0.21.8. The official Dagger Actions integration is pinned to v8.4.1's commit. Only the check scripts and the sources and tests of native/, model/ and the guide optimiser enter the container; local environments, experiments and build outputs are filtered. See Dagger's directory filtering.

Both the Actions runners and the Dagger container install nextest with tools/install_nextest.sh, which downloads the 0.9.143 release binary for the platform and verifies it against a SHA-256 digest pinned in the script. Compiling nextest from source took between two and twelve minutes per job and was most of each run. Updating nextest means changing the version and the digests together; the script header shows how to list them. No build cache is used.

Checks cover formatting, Clippy with warnings denied, nextest with four workers, doctests, build and CLI help. Every command retains a log and exit code. evidence exports these even when a check fails; check enforces the aggregate exit code. Actions runs both in order and uploads evidence with if: always(). Setup or container failures remain failed jobs, even if no evidence directory could be produced. Artifacts are retained for 14 days. Dagger may reuse execution results for identical content and arguments; source changes or a different revision invalidate the check execution. The enforcement call reuses the same evidence result rather than running the suite twice. Cargo dependency/build caches are separate from this content-addressed execution cache.

Three checks keep the core free of models (spec model-seam, task 7.4):

  • separation-suites runs before the core's suite, which holds the step flow's and the driver's tests, and fails when it finds a lesscode-model: LESSCODE_MODEL_BIN set, one in the core's debug directory or its deps, or one on the search path. It runs no test itself.
  • separation-lock fails when native/Cargo.lock names genai (the seam's client library), reqwest or hyper (the HTTP client and implementation it brings), ureq (the core's client before the seam) or lesscode-model.
  • separation-model-source runs the source scan in model/tests/call.rs under its own name; it fails on a model name of the first cases or an address under model/src.

The seam itself is checked by five more steps: model-fmt, model-clippy, model-test (its suite against a stub endpoint; no model is called), model-doctest and model-budgets, which measures model/src against model/complexity-budgets.json with the core built earlier in the script.

These five, separation-model-source and the guide optimiser's steps (format, lints, two dry runs with a scripted stand-in) run in the Linux job only (owner decisions, 2026-11-16): the Dagger container sets LESSCODE_CHECK_SEAM=1 and mounts the one guide file the dry run reads. No other platform compiles the client library. The seam and the optimiser share one target directory (model/target). Before this, the library's dependency tree was compiled four times per platform, about eight of nineteen minutes on Windows (run 43903666288).

native/.config/nextest.toml and model/.config/nextest.toml end a test that runs longer than five minutes and name it as timed out. The slowest test took 115 seconds on a hosted runner when the limit was set.

The first hosted run 41333477507 passed all three jobs: Linux 103 tests, Windows 102 and macOS 103, with zero skips. All check exit codes were zero. The separate doctest command passed with zero doctests defined. The durable audit records the PR merge checkout, matching reviewed native tree and artifact IDs/digests. Local WSL Dagger also passed, including an intentional failure probe that confirmed evidence export and nonzero enforcement; see the local receipt. Its pending-Actions label describes the earlier capture time, superseded by the hosted audit.

Multi-platform and multi-architecture CI matrix

.github/workflows/native-core.yml runs the native checks on five platforms:

Platform Where it runs
Linux x86_64 In a container through Dagger (rust:1.98.1-bookworm)
Linux aarch64 GitHub-hosted ubuntu-24.04-arm
macOS Apple Silicon GitHub-hosted macos-15
macOS Intel GitHub-hosted macos-15-intel
Windows x86_64 GitHub-hosted windows-2025

Step guard on this repository

The job Step guard in the same workflow builds lesscode-core from the pushed commit and runs

lesscode-core step guard --branch "$BRANCH" --revision HEAD

A branch that a step declaration names passes only with an accepted verdict, or an override on the ledger, for exactly the content it has (steps.md). Any other branch, main included, passes as "not a step branch". The checkout fetches the whole history, because the guard reads the base commit of the step. To make the job a boundary, mark it as a required check for main in the repository settings; the workflow alone only reports.

Adopter check: retired

tools/adopter_check.sh and tools/adopter-check.compose.yaml installed a released binary in a clean container and ran the documented commands. They were written for release 1.0.x and were removed on 2026-11-12 (owner decision); the install check of the onboarding skill is to take their place. The last version is in the history before that date.

Release binaries

.github/workflows/release.yml builds and signs the adopter binaries. Pushing a tag of the form vX.Y.Z runs two stages:

  1. build compiles lesscode-core (native/) and lesscode-model (model/), each from its own lock file, with Rust 1.98.1 and --locked on the same five runners as the correctness matrix, one per target: x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu, x86_64-pc-windows-msvc, x86_64-apple-darwin, aarch64-apple-darwin. Each job runs each built binary once (--version), packs it with LICENSE and its crate's LICENSES.md and THIRD_PARTY_NOTICES.txt into lesscode-core-<target>.tar.gz and lesscode-model-<target>.tar.gz (.zip on Windows) and keeps the two unsigned archives as one workflow artifact. These jobs hold no secrets and cannot write to the repository.
  2. publish runs only if all five builds succeeded, in the release environment, so it waits for the owner's approval. It checks that the tag equals the version in native/Cargo.toml, installs zipsign 0.2.2, signs every archive, writes the .sha256 files and SHA256SUMS after signing, signs SHA256SUMS, attests build provenance, uploads all files together with scripts/install.sh and scripts/install.ps1 to a draft release and then publishes it as latest. It fails closed without the signing key or when the key is not listed in scripts/release-public-keys.txt. Release signing has the owner setup and how to verify.

Asset names carry no version, so https://github.com/norandom/lesscode/releases/latest/download/<asset> always resolves to the newest release. The install scripts rely on that.

Starting the workflow by hand (workflow_dispatch) runs only the build stage. Use it to check packaging before tagging. The release workflow does not run the test suite; tag only a commit whose Native core correctness run is green. The Linux binaries link against the runner's glibc (2.39 on Ubuntu 24.04).

To cut a release: bump the version in native/Cargo.toml, native/Cargo.lock, CITATION.cff, .zenodo.json and the README badge; push; wait for green CI; push the tag; approve the release environment when asked. Zenodo archives the release when it is published. The tag follows the core's version; lesscode-model states its own version from model/Cargo.toml.

When a lock file changed, rebuild the license inventory the archives carry before tagging: ./scripts/license-inventory.ps1 -Crate native and -Crate model write LICENSES.md and THIRD_PARTY_NOTICES.txt from locked Cargo metadata; with -Check the script only fails when a file would change. A package that publishes no license file needs its text under the crate's licenses/ first; the script names it.

Historical Python baseline (Archived)

The Python prototype (lesscode-baseline) and its corresponding .github/workflows/baseline-platforms.yml workflow were retired and archived at commit fd0b5b9. The historical evidence and reports remain preserved in evidence/. runtime targets or environment configuration during integrity verification.

tools/ci_validate.py retains commands, working directories, exit outcomes, stdout/stderr, timings and runtime identities. The helper supplies a canonical dedicated scratch directory through TMPDIR, TEMP and TMP; its path is recorded in runtime.json. This avoids macOS's default /var alias while preserving strict source and artifact symlink guards. Metric staging independently canonicalizes its owned temporary root before launching analyzers, keeping Windows 8.3 paths and actual process cwd consistent. It runs canonical verbose unittest discovery with absolute source PYTHONPATH, followed by the native parent/descendant timeout test. Host-inapplicable unittest skips remain visible in the canonical log. Other failures remain failures. A failed test stage does not prevent independent build and installed-package evidence from being gathered.

The same job builds the wheel, creates a fresh uv environment outside the checkout, installs the wheel without dependencies, and checks both console and module help/version from that external directory with PYTHONPATH absent. It then runs tools/validate_pilot.py: six genuine evaluations, repeated structural metrics, known complexity/duplication values, preserved originals, behavior rejection and experiment aggregation. Completed pilot recovery uses --resume and verifies the retained artifacts read-only. Partial pilots require a fresh output directory and cannot become a resumed success.

The baseline-<runner>-<attempt> artifact is uploaded even after a failed stage. It retains bootstrap logs, validation/commands.json, runtime.json, native test diagnostics, probe results, the wheel and pilot artifacts for 14 days. The disposable installed environment is excluded. GitHub job logs also retain action setup failures. Analyzer dependencies are locked; only uv's dependency cache is enabled. Evaluations and result artifacts are never cached. Uploaded analysis inputs are exclusively the checked-in public fixtures; there are no model calls or uploads of user projects.

Notice files use targeted Git -text attributes because their names identify exact distribution bytes, including upstream CRLF. The toolchain tests verify all retained notice hashes and Git's effective attribute. Preserve those bytes when changing the inventory; never normalize an upstream notice to repair its filename hash.

A green workflow configuration or a local Windows check is insufficient cross-platform evidence. Read the native job outcomes, canonical skips, validation/result.json and pilot validation.json for the tested revision. An unexecuted platform remains outstanding. This validates the harness on the declared fixtures; it does not establish general simplification effectiveness.

Action SHA pins were resolved from official upstream tags on 2026-10-31:

Runner labels are documented in the GitHub-hosted runner reference.

The independently reviewed native run 41198054658 tested exact runtime revision 778e46258dbca42d5b07191a4f011b2cb858fb61. All three jobs passed 363 tests each and all 51 stages passed. The durable compact audit summary retains exact job and artifact IDs, archive digests, skips and measured pilot outcomes. Raw workflow artifacts have 14-day retention and may expire; rerun the pinned workflow at the desired revision to regenerate inspectable raw evidence. See integrated readiness for runtime equivalence, fresh installed validation, all-criteria coverage and remaining review gates. This record establishes fixture-level evaluator validation only.