Skip to content

TEMPORARY: quarantine the macOS Accelerate gesdd test (revert after Mark's BLAS++/LAPACK++ review) - #157

Merged
mmelnich merged 2 commits into
mainfrom
macos-quarantine-accelerate-gesdd
Aug 12, 2026
Merged

TEMPORARY: quarantine the macOS Accelerate gesdd test (revert after Mark's BLAS++/LAPACK++ review)#157
mmelnich merged 2 commits into
mainfrom
macos-quarantine-accelerate-gesdd

Conversation

@mmelnich

@mmelnich mmelnich commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Warning

This is a temporary change and is meant to be reverted.
It is one clean git revert of a single commit.

Revert when these land

The quarantine exists only until Apple's new Accelerate interface is in use. Both are open and awaiting review:

PR Status
icl-utk-edu/blaspp#134 — New Apple Accelerate support (#74, rebased and completed) open
icl-utk-edu/lapackpp#88 — Support Apple's new Accelerate interface (closes #43) open

Order matters: lapackpp#88 depends on the defines.h from blaspp#134, so #134 merges first. blaspp#134 is a rebased and completed continuation of the stalled blaspp#74.

Once both are merged, RandLAPACK#155 (retire legacy-Accelerate accommodations) becomes mergeable and TestQB.Polynomial_Decay_general1 should pass on its own — at which point the warning added here fires and tells you to delete this quarantine.

Why

TestQB.Polynomial_Decay_general1 fails on Apple Silicon because Apple's old Accelerate LAPACK has a broken divide-and-conquer gesdd. It has been left failing deliberately, as a canary for the planned migration.

The side effect is that core-macos has been permanently redbuild and build-asan, on main and on every branch. A lane that is always red carries no information: you cannot distinguish a new macOS regression from the known one, so the job gets ignored and the canary signals to nobody.

What this does

Quarantines the test without discarding the signal:

  • The main ctest run excludes it, so the job reflects real macOS health.
  • The test is then still run, separately, and cannot fail the job.
  • Every macOS run prints a warning that the suppression is active and names the condition for removing it.
  • If the test ever passes, the job prints a louder warning saying to delete the quarantine.

That last point matters: the migration signal is preserved and actually improved. Previously it was "a red job turns green", which nobody was watching because the job was always red. Now it is an explicit annotation telling you what to delete.

Scope

Deliberately surgical — core-macos.yaml (both jobs) and the corresponding docs/CI.md entry, nothing else, so the revert stays clean. The dead TestABRIK.ABRIK_catch_instability exclusion in these same lines is left alone for that reason.

…an go green

REVERT THIS ONCE MARK HAS REVIEWED THE BLAS++ / LAPACK++ ACCELERATE WORK.

TestQB.Polynomial_Decay_general1 fails on Apple Silicon because Apple's old
Accelerate LAPACK has a broken divide-and-conquer gesdd. It was left failing
on purpose, as a canary for the planned migration to the new Accelerate
interface.

The problem with that arrangement is that core-macos has been permanently red,
across build and build-asan, on main and on every branch. A lane that is always
red carries no information: nobody can tell a new macOS regression from the
known one, so in practice the job is ignored and the canary signals to no one.

This quarantines the test rather than deleting the signal:
- The main ctest run excludes it, so the job reflects real macOS health.
- The test is then run separately and cannot fail the job.
- Every run prints a warning that the suppression is active, naming the
  condition for removing it.
- If the test ever PASSES, the job prints a louder warning saying to delete the
  quarantine. That is the migration signal the old always-red setup was meant
  to give, except it now arrives as a specific instruction instead of a colour
  change nobody was watching.

Deliberately surgical: nothing else in these files is touched, so reverting is
a clean revert of this one commit.
…macOS quarantine

blaspp#134 and lapackpp#88 (in that order -- #88 needs #134's defines.h), plus
the stalled blaspp#74 they continue. Without the PR numbers, 'revert once the
Accelerate work lands' is not actionable by anyone who was not in the room.
@mmelnich

Copy link
Copy Markdown
Contributor Author

Tracked for revert in #159 — the revert is to happen in #155, once blaspp#134 and then lapackpp#88 are approved and merged.

@mmelnich
mmelnich merged commit 4cc72a8 into main Aug 12, 2026
22 of 24 checks passed
@mmelnich
mmelnich deleted the macos-quarantine-accelerate-gesdd branch August 12, 2026 20:04
mmelnich added a commit that referenced this pull request Aug 12, 2026
## Problem

A Windows install failed on a plain Visual Studio machine. Fixing it
uncovered **three independent causes**, all invisible to CI for the same
reason: CI never runs the configuration a user actually has.

**1. vcpkg.** The installer ran a classic-mode `vcpkg install`, but the
vcpkg bundled with Visual Studio is manifest-only. CI runners ship a
separate, classic-capable vcpkg.

**2. A 32-bit toolchain.** BLAS++ reported `BLAS library not found`
while oneMKL was correctly installed *and discovered*. The real cause is
one line earlier: `.../bin/Hostx86/x86/cl.exe` — a 32-bit compiler,
whose linker cannot use an x64 import library.

Our documentation caused it: it said to open "Developer PowerShell for
VS 2022", which defaults to **x86**. "Developer Command Prompt" is a
*64-bit process* that also defaults to x86, so shell bitness is not a
usable signal — only "x64 Native Tools Command Prompt" gives an x64
toolchain. CI sets `arch: x64` explicitly, so it never ran the
prescribed shell.

**3. Spaces in the library path.** With a correct x64 toolchain *and*
our own link check passing, BLAS++ **still** failed. Its `try_compile`
log gives the reason:

```
ninja: error: 'C:/Program', needed by 'cmTC_x.exe', missing and no known rule to make it
```

BLAS++ flattens `BLAS_LIBRARIES` from a CMake list into a
space-separated string, then splits it back on spaces before probing.
That round-trip is lossy: once joined, a space *inside* a path is
indistinguishable from a separator. Intel installs oneMKL to `C:\Program
Files (x86)\...` by default, so **every discovered oneMKL hits this**.
It stayed hidden because the *downloaded* oneMKL lands in a space-free
directory — the layout CI uses.

## Library discovery and provisioning

Windows has **no system prefix for third-party libraries, no loader
cache, no RPATH**. So build-time discovery cannot be a filesystem search
(CMake's own `FindBLAS` finds MKL only via `MKLROOT`), and at run time
Windows searches the executable's own directory **first** and `PATH`
**last**. PATH is the wrong tool in both phases.

**`-Backend mkl` (default)** is the one backend with real discovery,
since oneMKL has a canonical location. Probed in order, first match
wins:

| # | Source |
|---|--------|
| 1 | `-MklRoot <path>` — explicit; invalid is a hard error, never a
silent fallback |
| 2 | `$env:MKLROOT` — set by `setvars.bat`; the variable CMake's
`FindBLAS` uses |
| 3 | `$env:ONEAPI_ROOT\mkl\latest` |
| 4 | `C:\Program Files (x86)\Intel\oneAPI\mkl\latest` — the installer
default |

A candidate counts only if it holds `mkl_intel_ilp64_dll.lib` under
`lib\` or `lib\intel64\` alongside a DLL directory (`bin\`, or
`redist\intel64\` pre-2024), so a partial install is rejected rather
than half-used. If nothing is found the installer states what it
searched and **asks** before downloading a pinned, checksum-verified
copy into the project directory; declining lists the alternatives and
exits non-zero. `-NoDownload` turns "not found" into an error outright.

**`-Backend openblas`** gets no discovery deliberately — no canonical
location exists (GitHub zips, vcpkg, conda and MSYS2 all differ, and the
release zips ship configs with wrong hardcoded paths). It asks whether
you already have OpenBLAS and, if so, prints the exact `-Backend custom`
invocation.

**`-Backend custom`** takes your `.lib` paths and DLL directory — the
route for AMD AOCL, whose downloads are licence-gated.

Whatever the source, libraries are proved to work **before any
dependency is built**, by compiling, linking and *running* a
`dgemm_`/`dgesv_` program with a numeric check. That check previously
skipped the default `mkl` backend, which is why cause 2 surfaced three
layers down as a misleading BLAS error. Where paths contain spaces,
import libraries are staged into a space-free directory. That is a
workaround: the underlying bug is fixed upstream in
[icl-utk-edu/blaspp#137](icl-utk-edu/blaspp#137),
and removing the staging once that lands is tracked in #158.

Prompts appear only when someone can answer them: `$script:Interactive`
is false whenever stdin is redirected, mirroring `install.sh`'s
`INTERACTIVE` flag. Every question has a defensible unattended default,
so CI cannot hang.

## What this PR does

1. **vcpkg removed.** oneMKL comes from Intel's official NuGet packages,
pinned by version and SHA256. No package manager prerequisite.
2. **Backend choice** — `-Backend mkl|openblas|custom` — with the
discovery, provisioning and validation above, plus `-NoDownload` /
`-Yes`.
3. **x64 toolchain guard** in both `install.ps1` preflight and
`setup.ps1`, with *different* messages for x86 (wrong shell, one-command
fix) and arm64/arm (unsupported — no oneMKL build exists). Shared via
`.github/scripts/windows/toolchain-arch.ps1`; reads
`VSCMD_ARG_TGT_ARCH`, then the `bin\Host<host>\<target>\` convention,
then `cl.exe`'s banner — the banner alone would miss on a localized
Visual Studio, and a missed detection fails *open*.
4. **No PATH edits, ever.** Runtime DLLs are staged beside each
executable (app-local deployment). `RANDLAPACK_RUNTIME_DLL_DIRS` covers
the backend DLLs `TARGET_RUNTIME_DLLS` cannot see; the installed package
exports `randlapack_stage_runtime_dlls()` for downstream projects.
5. **MSVC OpenMP fixed, and now reaching users.** RandLAPACK's
`find_package(OpenMP)` ran before RandBLAS's `/openmp:llvm` guard, so
classic `-openmp` was cached — under which MSVC silently ignores the
`collapse` clause `rl_rpchol` needs (C4849). The installer also
unconditionally disabled OpenMP, so the fix shipped unusable; it is now
on by default, with `-NoOpenMP` to opt out.
6. **Dependencies from upstream, pinned to immutable refs.**
BLAS++/LAPACK++ came from forks carrying two one-line MSVC fixes; both
merged upstream 2026-08-06 (icl-utk-edu/blaspp#132,
icl-utk-edu/lapackpp#87), so both now come from `icl-utk-edu`. They were
pinned to *branch names* inside a cache keyed on the setup script, so a
cache hit could restore a different revision than a miss builds.
Everything the installer can fetch is now the **newest stable release,
pinned to an exact version**: oneMKL `2026.1.0.226`, OpenBLAS `0.3.34`,
GoogleTest `v1.18.0`, Random123 `v1.14.0`. BLAS++ `3057185` and LAPACK++
`40b9d0d` are the two exceptions, pinned to commits because the latest
release of each (`v2025.05.28`) predates the MSVC fixes; they move to a
tag once one carries them. The oneMKL bump renames the runtime DLLs
`mkl_*.2.dll` -> `mkl_*.3.dll`; nothing hardcodes those names (staging
globs `*.dll`), and both provisioning paths were re-verified after the
bump.
7. **Reuse gated on provenance, not presence.** A clone is reused only
if at the pinned remote and ref; a built dependency only if built from
the source we would build from now. Without this, changing a pin is a
no-op for anyone who already has an install.
8. **One CI job, `windows-toolchain-guards`,** covering the documented
user path the build matrix structurally cannot. It asserts refusals and
runs pure logic, so nothing builds and it finishes in seconds. A
decision table covers `arm64`/`arm` — the only possible coverage, since
we cannot build for them — and integration steps run the installer under
a real x86 toolchain and an `amd64_arm64` cross-compiling one, giving an
arm64-targeting `cl.exe` on an x64 runner. It launches exactly as the
docs prescribe, so the documented invocation stays under test.
9. **CI ran every job twice.** All four workflows fired on
`pull_request` *and* on `push` for every branch, so each commit ran the
full matrix twice — same SHA, same result, 22 check runs where 11 would
do. `push` is now restricted to `main`. Nothing cancelled superseded
runs either, so pushing a fix left the previous run going to completion;
a `concurrency` group now supersedes in-flight runs, for pull requests
only (on `main` every commit should still be validated). This removes
automatic CI for a branch with no pull request open, which
`workflow_dispatch` covers on demand.
10. **Smaller fixes:** Ninja unified across dependency builds;
`--retry-all-errors` on downloads (plain `--retry` misses
connection-level failures like curl 52); cache keys off `hashFiles()`,
which silently resolves EMPTY under the install-script checkout path and
split caches that claimed to be shared; and a dead `ctest` exclusion for
`TestABRIK.ABRIK_catch_instability` — a test gone since January 2025 —
removed from the Windows paths, where it had leaked into the user-facing
installer even though `install.sh` has no such exclusion.
11. **Docs.** New [INSTALL_WINDOWS.md](INSTALL_WINDOWS.md): quick start,
Windows-vs-Unix contrast, backend table, `install.ps1` reference,
runtime-DLL explainer, troubleshooting. It states the toolchain contract
the way Linux and macOS do -- you bring a compiler, CMake, Ninja and
Git, in whatever terminal you like -- and then offers ways to satisfy it
rather than mandating one shell, since naming a single Start-menu entry
without naming the requirement is what let the wrong shell go unnoticed.
It verifies with `where cl` (bare `cl` is silent in both failing cases),
gives a `vswhere -latest -products *` one-liner that works for any
edition (without `-products *` it finds nothing on Build Tools), and
documents `-ExecutionPolicy Bypass`, since stock Windows refuses to run
`.ps1` at all.

## Verification

All Windows CI green. Every backend and both interactive branches were
also exercised on a machine that began with **no Visual Studio, CMake,
Git, Ninja or MKL**, under Windows PowerShell 5.1 rather than CI's
PowerShell 7, and later against a real oneAPI install at the default
(spaced) location:

| Path | Result |
|---|---|
| oneMKL **discovery** (the failing real-world case) | 749/749 |
| oneMKL downloaded, from scratch | 745/745 serial, 749/749 OpenMP |
| Build from the **upstream pins** | 749/749, origins confirmed
`icl-utk-edu` |
| `-Backend openblas` | 745/745 |
| `-MklRoot` valid / invalid | reused / hard error |
| `-Backend custom`, insufficient library | rejected (`LNK2019:
unresolved dgemm_`) |
| openblas interactive yes / no | custom recipe / downloads |
| `-NoDownload` (mkl, openblas, declined) | all error, nothing fetched |
| Real x86 toolchain | refused at preflight |
| Guard decision table | 5/5, plus a mutation test catching a sabotaged
guard |
| Stale provenance | rebuilds; reuses on second run; fork clone
re-cloned |
| After bumping oneMKL to 2026.1.0.226 and GoogleTest to v1.18.0 |
discovery 749/749; forced NuGet download verified both SHA256s and
staged the renamed `mkl_*.3.dll` |

The rows above the last one were measured before the version bump; the
bump was then re-verified on both provisioning paths, which is the last
row.

Rebased onto `main` after #157, which quarantined the macOS Accelerate
`gesdd` canary, so `core-macos` is green here rather than carrying a
known failure.

## Notes for reviewers

- The `amd64_arm64` leg is the only thing exercising a genuine ARM64
compiler; it could not be verified locally (adding the toolset needs
interactive elevation) and passed on its first run.
- The space-in-path staging works around a BLAS++ quoting bug; fixing
that upstream would remove the need for it.
- Pre-existing and left for separate changes: `install.sh` clones
BLAS++/LAPACK++ at floating HEAD, and the same dead `ctest` exclusion
remains in the three Linux/macOS workflows.
mmelnich added a commit that referenced this pull request Aug 13, 2026
…161)

## Problem

Two unrelated CI defects, both of which fail quietly rather than loudly.

**The Windows oneMKL cache has never hit.** `action.yml` pointed `path`
at `windows-deps\onemkl-2025.2.0.627`, while `setup.ps1` creates
`onemkl-$mklVersion` with `$mklVersion = "2026.1.0.226"`
(`setup.ps1:384,390`). The path was left behind when the version was
bumped, so it names a directory that does not exist.

Nothing reports an error. `actions/cache` looks up a key, misses, the
job re-downloads **155 MB** of oneMKL, and then the post-job step saves
a cache entry for a path that isn't there. So the cache can never hit,
on every MKL leg, indefinitely. The key already said `2026.1.0.226` —
only the path was stale.

**A `ctest` exclusion for a test that does not exist.** `--exclude-regex
"^TestABRIK\.ABRIK_catch_instability"` appears at six sites. That test
is nowhere in `test/` or `RandLAPACK/`, and `git log -S` finds no
history of it there under that name, so it has always been excluding
nothing.

The cost isn't the wasted filter. It's the misdirection: a `ctest` line
carrying an exclusion reads as though a known failure is being
suppressed, which invites exactly the wrong conclusion when someone
audits what the suite actually covers.

## What this PR does

1. **Corrects the cache path** to `onemkl-2026.1.0.226`, and adds a
comment recording that the version there must track `$mklVersion` in
`setup.ps1`. Every other path/key pair in that file was checked and is
consistent — OpenBLAS, GoogleTest, GoogleTest-ASan, Random123, BLAS++
and LAPACK++ all agree.
2. **Removes the dead exclusion** from all six sites: `core-linux.yaml`
(×2), `install-script.yaml` (×2), `core-macos.yaml` (×2).

## Notes for reviewers

- **The macOS change is deliberately partial.** There the regex was an
alternation with `TestQB.Polynomial_Decay_general1`, which is the *real*
Apple Accelerate `gesdd` quarantine from #157. Only the dead ABRIK half
is removed; the quarantine, its "suppression is ACTIVE" warning, and the
check that shouts if the test starts passing are all untouched.
- Expect the first Windows MKL leg after this merges to still be slow —
it has to populate the cache once. Subsequent runs are where the 155 MB
comes back.
- This is the first of a short series bringing RandLAPACK's `install.sh`
up to the standard `install.ps1` reached in #156, mirroring the RandBLAS
installer work. It is independent of the rest and useful on its own.
mmelnich added a commit that referenced this pull request Aug 13, 2026
…kend selection

Brings the Unix installer up to the standard install.ps1 reached in #156, and
in line with the RandBLAS installer. The Windows script already pinned every
dependency, recorded provenance and validated the BLAS before building; this one
did none of that. Eleven defects, all verified against main before changing
anything:

* Three `git clone` calls with no ref (:336,339,342) -- blaspp, lapackpp and
  random123 all tracked upstream default branches, so two runs of the same
  script could build different source. Now pinned to the refs this repository's
  own Windows provisioner already validated, fetched one commit deep, with a
  provenance stamp on each install and each source tree.

* No BLAS validation at all. The installer now compiles, links and *runs* a
  program against the finished install, through BLAS++ and LAPACK++ rather than
  raw dgemm_ (that is how RandLAPACK reaches them), checking a gemm result and
  a gesdd factorization numerically.

  The gesdd call is deliberate. It is the routine Apple's legacy Accelerate
  computes incorrectly, so a broken SVD surfaces at install time rather than in
  someone's RSVD output. And the run-it-not-just-link-it part matters for a
  reason peculiar to integer width: BLAS++/LAPACK++ guard the int64_t downcast
  and throw rather than truncate, but that guard keys off sizeof(blas_int) as
  declared by the *header*. If the headers say 64-bit while the loaded library
  is LP64, the guard compiles out and 64-bit values reach routines reading 32
  bits -- which shows up not as wrong numbers but as an absurd workspace size
  and a run that dies in allocation or never finishes. Nothing catches that by
  inspection.

* /opt/homebrew hardcoded in eight places, which hard-fails on Intel macOS and
  any custom HOMEBREW_PREFIX. Now `brew --prefix`.

* The script moved the user's own clone into <project>/lib/RandLAPACK, breaking
  git worktrees. It now works in place and puts a symlink there instead, so the
  layout still reads as complete and the path CI invokes still resolves.

* The log was truncated on every run (:211), destroying the previous run's
  output at exactly the moment you want it. Now appended with a run header.

* The libdir search omitted lib/aarch64-linux-gnu.

* No --prefix; extras and benchmarks had no way to be skipped. Both added,
  with extras and benchmarks still built by DEFAULT (--no-extras /
  --no-benchmarks to opt out): unlike RandBLAS's examples they need nothing this
  script has not already built, so default-on costs only time.

Backend selection is new: --blas=auto|openblas|mkl|accelerate|custom,
--blas-int, --blas-libraries. Integer width is requested per backend and then
READ BACK from BLAS++'s generated blas/defines.h rather than assumed, because
BLAS++ probes int32 before int64 while blas_int only filters library *names* --
so for MKL the request genuinely selects mkl_intel_ilp64, but for OpenBLAS
there is only -lopenblas and an LP64 build passes the int32 probe and is
accepted. Trusting the request would stamp LP64 installs as ILP64.

macOS deliberately keeps Homebrew OpenBLAS as the default rather than moving to
Accelerate. Apple's legacy Accelerate has a broken divide-and-conquer gesdd and
RandLAPACK calls gesdd in rl_rsvd, rl_abrik, rl_revd2, rl_preconditioners and
rl_util, so Accelerate would quietly return wrong singular values across most
of the SVD-based drivers -- which is what #157's quarantine of
TestQB.Polynomial_Decay_general1 is about. --blas=accelerate is allowed and
warns, citing #159.

The provenance stamp includes the GPU setting, and the dependency install
directories now carry the backend and GPU configuration in their names
(blaspp-mkl-cpu-install). Both matter, and the second one for a reason outside
this project: RandBLAS's installer uses the same RandNLA-project layout and
named its BLAS++ install identically. Sharing that name meant this script would
rebuild over RandBLAS's BLAS++ (their stamp is a different file, so it never
matched ours), and RandBLAS's next run would reuse an artifact we had replaced
underneath it while its own stamp still described the original. Distinct names
make that impossible; cross-project sharing is instead explicit, via
BLASPP_INSTALL_DIR, with the conftest confirming the result works.

Also adds the three progress tiers from the RandBLAS installer -- determinate,
parsed from Ninja's "[12/34]" and Make's "[ 42%]" -- with tier 0 (redirected
output) byte-identical to the plain step list, and a CI assertion that
redirected output contains no escape sequence or carriage return.

CI: the install-script lanes globbed rather than hardcoded for the new
dependency directory names, so they are not coupled to that naming, and the
discovery assertion updated to the text the new script prints.

Verified on Linux (gcc 15.2, oneAPI MKL): fresh MKL build; idempotent re-run;
full default run with extras and benchmarks (13/13 steps); switching --no-gpu to
--gpu correctly invalidating the BLAS++ provenance and rebuilding; dependency
discovery reducing to 3 steps; running from a git worktree with the clone left
in place; redirected output escape-free; and RandBLAS-then-RandLAPACK into one
shared RANDNLA_PROJECT_DIR leaving both projects' dependencies intact, plus
RandLAPACK reusing RandBLAS's BLAS++ when pointed at it explicitly.
mmelnich added a commit that referenced this pull request Aug 13, 2026
… backend selection, and docs (#162)

## Problem

RandLAPACK's two installers had drifted apart. `install/install.ps1` was
rebuilt in #156 and pins every dependency, checksums downloads, records
provenance and link-and-run validates the BLAS before building anything.
`install/install.sh` did none of that, and the documentation described a
script that no longer existed in several respects.

Eleven defects, all verified against `main` before touching anything:

| Where | Defect |
|---|---|
| `install.sh:336,339,342` | three `git clone` with **no ref** — blaspp,
lapackpp, random123 all tracked upstream default branches |
| — | **no BLAS validation at all** |
| — | `/opt/homebrew` hardcoded, **8 places** — hard-fails on Intel
macOS and custom `HOMEBREW_PREFIX` |
| `install.sh:331` | `mv "$REPO_DIR"` relocated the user's own clone,
breaking git worktrees |
| `install.sh:211` | `: > "$LOG"` truncated the log, destroying the
previous run's output |
| `install.sh:247` | libdir search omitted `lib/aarch64-linux-gnu` |
| — | no `--prefix`; extras and benchmarks unconditional with no way to
skip |
| `install.ps1` | **zero** references to `RANDNLA_PROJECT_DIR`, no
`--modify-rc` equivalent |
| `INSTALL_SCRIPT.md` | documented the pre-backend flag list, and still
told people the script would move their clone |

## The two things most worth reviewing

### Integer width is read back, not assumed

BLAS++ probes `int32` before `int64`, and `blas_int` only filters which
*library names* to try. For MKL that is a real choice — `mkl_intel_lp64`
and `mkl_intel_ilp64` are different libraries. For OpenBLAS there is
only `-lopenblas`, so **a successful `blas_int=int64` configure proves
nothing**: an LP64 build passes the `int32` probe and is accepted. The
resolved width therefore comes from BLAS++'s generated `blas/defines.h`
after the build. Trusting the request would stamp LP64 installs as ILP64
and have every later run reuse them believing otherwise.

### Verification runs the program, and that is the point

The conftest compiles, links and *runs* against the finished install —
through BLAS++ and LAPACK++ rather than raw `dgemm_`, since that is how
RandLAPACK reaches them — checking a `gemm` result and a `gesdd`
factorization numerically. `gesdd` is chosen because it is the routine
Apple's legacy Accelerate computes incorrectly.

Running it rather than only linking matters for a reason specific to
integer width. BLAS++ and LAPACK++ *do* guard the `int64_t` downcast —
`to_blas_int` (`blaspp/src/blas_internal.hh:16-22`) throws rather than
truncates — but that guard keys off `sizeof(blas_int)` **as declared by
the header**. If the headers say 64-bit while the library actually
loaded is LP64, the guard compiles out and 64-bit values reach routines
reading 32 bits. That surfaces not as wrong numbers but as nonsense
control values: a misread workspace query becomes an absurd `lwork`, and
the run dies in allocation or never finishes. Nothing catches it by
inspection.

## A cross-project collision, found while testing

Dependency install directories now carry backend and GPU in their names
(`blaspp-mkl-cpu-install`). That is not cosmetic.

RandBLAS's installer uses the same `RandNLA-project` layout and named
its BLAS++ install **identically** (`blaspp-<backend>-install`), with a
different stamp filename. With a shared `RANDNLA_PROJECT_DIR`, this
script would rebuild over RandBLAS's BLAS++ — their stamp never matches
ours — and **RandBLAS's next run would reuse an artifact we had replaced
underneath it, while its own stamp still described the original.**
Distinct names make it impossible; sharing is instead explicit via
`BLASPP_INSTALL_DIR`, verified by the conftest.

## Everything else

- **macOS keeps Homebrew OpenBLAS as the default.** Apple's legacy
Accelerate has a broken divide-and-conquer `gesdd`, and RandLAPACK calls
`gesdd` in `rl_rsvd.hh:146`, `rl_abrik.hh:692`, `rl_revd2.hh:208`,
`rl_preconditioners.hh:355` and `rl_util.hh:413`. That is what #157's
quarantine is about. `--blas=accelerate` warns, citing #159. The
hardcoded `/opt/homebrew` is fixed regardless.
- **Extras and benchmarks stay built by default** (`--no-extras` /
`--no-benchmarks`), unlike RandBLAS's examples — they need nothing this
script has not already built.
- **The clone is never moved**; a symlink gives the same layout and the
path CI invokes still resolves.
- **RandBLAS is untouched** — still a pinned submodule, still
authoritative.
- **`install.ps1`** now honours `RANDNLA_PROJECT_DIR` with the same
precedence as `install.sh`, gains `-ModifyEnvironment` (User-scope, the
only thing that survives a new shell) and `-Prefix`, and its path-length
warning now checks the resolved path rather than only an explicitly
passed one.
- **Progress rendering** in three tiers, determinate from the build
tool's own output, with tier 0 byte-identical to the plain step list —
plus a CI assertion that redirected output carries no escape sequence.
- **`INSTALL_SCRIPT.md`** rewritten: current flags, corrected layout
diagram, and a new section 6 with a tested-configuration table drawn
from the CI lanes, the per-backend integer width, whether LP64 actually
limits RandLAPACK, and why macOS is on OpenBLAS.

## Verification

Local Linux (gcc 15.2, oneAPI MKL, NVIDIA GPU present):

| Scenario | Result |
|---|---|
| Fresh MKL build, `--no-gpu` | 9/9 steps, ILP64 confirmed from
`blas/defines.h` |
| Re-run in place | all six dependency steps reused, 1s rebuild |
| Full default run (extras + benchmarks) | 13/13 steps |
| `--no-gpu` → `--gpu` | BLAS++ provenance invalidated, rebuilt rather
than reused |
| Dependency discovery | all three reused, 3/3 steps |
| Run from a git worktree | clone left in place, worktree still
functional |
| Output redirected | no ANSI escapes, no carriage returns |
| **Full test suite** | **313/313 pass** |
| RandBLAS then RandLAPACK, shared `RANDNLA_PROJECT_DIR` | both
dependency sets intact |
| RandLAPACK reusing RandBLAS's BLAS++ | reused, only LAPACK++ built,
5/5 steps |

Windows 11, Windows PowerShell 5.1, VS 2022 Build Tools: all three
`-ProjectDir` precedence cases resolve as intended, and the User-scope
environment write round-trips.

## RandBLAS submodule bump

Also bumps the vendored RandBLAS from `04f2018` to `952251c` (RandBLAS
main), and `RandLAPACK_RandBLAS_PIN` with it.

The three commits in that range are **#185** (vcpkg manifest fix),
**#186** (CMake hygiene) and **#187** (installer scripts). None of them
touch `RandBLAS/` headers — the library code is byte-identical, and
everything in the range is build machinery. Two pieces of it matter
here:

- RandBLAS's CMake floor moved 3.12 → 3.21. RandLAPACK already declares
3.21, so nothing to change; the submodule now simply enforces what
RandLAPACK already required.
- RandBLAS exports `randblas_stage_runtime_dlls()` from its installed
package and prints a configuration summary. As a subproject under
RandLAPACK, which configures it with `BUILD_TESTS=OFF`, that summary
reports tests as deliberately skipped rather than warning.

`CMakeLists.txt` enforces that the pin variable and the submodule move
together, and it means it: the cross-check compares the variable against
`git ls-tree HEAD RandBLAS`, so a staged-but-uncommitted bump fails
configure until both land in one commit. Worth knowing if you ever bump
this iteratively — it is not satisfiable before committing.

Verified: clean build against the bumped submodule, and the full suite
re-run.

## Notes for reviewers

- **Consolidated from #162 + #163 + #164** at Max's request, matching
how the RandBLAS installer set was combined. Those two show as *merged*
rather than closed because this branch was fast-forwarded to contain
them — they merged into their stacked base, not into `main`. The tree is
byte-identical to the state tested and green as three separate PRs.
- Stacked on #161 (CI fixes), which stays separate — it is independent
and useful on its own.
- **`-ModifyEnvironment` is not verified end to end.** It runs at the
end of a successful install, and I could not reach it: the clone sits on
a `\\wsl.localhost` path and Windows `git` refuses it with `dubious
ownership`. Precedence and the User-scope write are each verified
directly, but not together in one completed run.
- Two upstream gaps are filed rather than worked around silently: BLAS++
never looks for an ILP64 OpenBLAS (#166), and implements only Apple's
legacy Accelerate interface (#165).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant