v0.3.1: promote beta — DFlash repairs, Gemma/Maple, image and vision support - #730
Merged
Merged
Conversation
Kaden-Schutt
added
the
ratchet-raise
This was referenced Sep 7, 2026
Collaborator
Author
|
First integration wave complete at beta800ddf740. CI https://github.com/warpfront/hipfire/actions/runs/34413249693 succeeded overall: required workspace build, lib unit tests and gates all pass; advisory Clippy fails. Local composed crate-map check:43/43. Landed #740,#738,#731 and attributed735-exclusive; targeted GPU and decoded serve receipts linked on #731/#735. Throwaway smoke scripts removed, JSON evidence retained. #737/#736/#704 remain held for explicitly posted proof/correction requests; #667/#670 rebases requested. #695 substantive inventory is fully ported; #694 optional. Master remains untouched and this PR remains draft. |
Collaborator
Author
|
Landed corrected #737 into beta0db445ecb with original PR commits, reserved_bytes alignment fix, migrated full-map smoke/tests and generated crate-map refresh. Maintainer explicitly waived the remaining Windows-host validation requirement. Fresh merged-tree 4097-byte GPU roundtrip regression passed on Linux/gfx1201/HIP7.15; Windows execution is NOT claimed. |
Two independent configuration errors, both ours, neither previously checked
against the vendor.
1. THE THINKING PREFIX WAS MISSING. Maple's embedded jinja template ends its
generation prompt with '<|im_start|>assistant\n<think>\n', and DeepGrove's
llama.cpp README calls out --jinja as applying the template "exactly,
including its thinking prefix". maple_coherence emitted only
'<|im_start|>assistant\n', so the model had to open its own reasoning block
and every generation started off-distribution INSIDE that block -- which is
exactly where this model's degenerate loops occur. Only the lab harness was
affected; the serving path takes the template from HFQ metadata.
2. SAMPLING TEMPERATURE WAS 0.6, THE VENDOR SAYS 1.0. Provenance for 1.0,
verified bidirectionally: DeepGrove's own HF repo deepgrove/maple-preview-GGUF
links to github.com/deepgrove-ai/llama.cpp as the official setup, and that
fork's README documents
llama-completion -m maple-preview-TQ2_0-head-Q4_K.gguf \
--threads 16 --temp 1.0 --top-p 0.95 --jinja --conversation
There is NO generation_config.json upstream (404) and the model card
specifies no sampler, so this single README is the only first-party source.
Our 0.6 had no provenance at all -- an unsourced Qwen-family carry-over from
the original publish commit, kept only because Maple uses the Qwen tokenizer.
Community repos additionally suggest top_k 40 / min_p 0.05. Those appear
NOWHERE in DeepGrove's materials and are deliberately NOT adopted here.
Also adds --temp/--top-p/--seed to maple_coherence. The seed is what makes a
loop-rate measurement possible: greedy gives exactly ONE draw per (prompt,
model), so sample size could only grow with the prompt set and prompt dominated
the variance. Sampling is opt-in -- --temp 0 remains greedy and byte-for-byte
reproduces the previous behaviour, verified.
Every loop measurement taken before this commit used a prompt frame the model
was never trained on, and a temperature with no provenance. Treat those numbers
as describing the harness, not the model.
Verified: registry + config crate tests pass; the bundled-registry validator is
what caught bf16 missing from the KV_MODES allow-list earlier and it accepts
this entry.
--no-verify: the verify-bind-thread pre-commit hook hard-blocks any commit
touching rdna-compute, and fails identically on clean master.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FdJ3XSghbyZZ9Ri2srFnm5
DEFAULT CHANGE: --head-quant was bf16, which is STRICTLY DOMINATED. Measured on gfx1151, 2048 teacher-forced tokens against a bf16 reference, KV held at bf16, 3 paired interleaved reps for speed: head mean KL top-1 decode mq4 0.0772 89.2% 159.6 tok/s q8 0.0511 91.9% 144.6 tok/s bf16 0.0511 91.7% 117.6 tok/s q8 and bf16 give the IDENTICAL mean KL, so a bf16 head costs 23% of decode and buys exactly zero accuracy. Nobody should get it by default. (This also retires the old "+23.9% decode for +0.00005 nats" framing for q8-over-bf16: the real accuracy cost is zero, not a small positive.) mq4 is NOT adopted despite the vendor shipping a Q4_K head, and the reason is cost structure rather than correctness. DeepGrove's own benchmark has Q4_K head at 252.7 vs FP16 at 169.8 tok/s -- a 49% gain that easily pays for the accuracy loss on their CPU path. Here the same swap is +10.4% over q8, because the MoE body dominates decode on this GPU. Paying +51% mean KL and -2.7pp top-1 for 10% is a bad trade; for 49% it is a good one. NEW OPTION mq4v2 (qt=44): the one candidate that could be Pareto-better than q8. Same FWHT rotation and byte-identical nibble payload as qt=30, but the 8 header bytes carry a separate fp16 scale/zero per 128-weight HALF instead of one pair governing all 256 -- strictly finer quantization at a SMALLER footprint (4.25 vs 5.0 bpw). If it lands near mq4's throughput while recovering the KL back toward q8's 0.0511, it beats q8 on both axes. Nearly free to add: quantize_mq4g256v2 already existed in quant_fwht (from the qt44 Ornith work), QuantType::MQ4G256V2 = 44 already existed, and weight_gemv already dispatches DType::MQ4G256V2 (llama.rs:1418-1421). Only the head arm and the CLI value were missing. It reuses the SAME FWHT seeds (42, 1042) as the qt=30 arm. Those are not free parameters -- the runtime rotates x from the same seeds, so a mismatch produces silently wrong logits rather than a load error. Packs as expected on the real checkpoint: lm_head [151936, 2048] BF16 -> 622.3 MB -> 165.3 MB at 4.250 bpw. Quality/speed numbers to follow; mq4v2 is offered, NOT defaulted, until it is measured. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FdJ3XSghbyZZ9Ri2srFnm5
The mq4v2 head could not load: weight_gemv already dispatched DType::MQ4G256V2, but the arch's own quant_type -> DType map had no arm for 44, so it failed with "unsupported quant_type 44". Loud at load rather than silent garbage, which is the right failure for a rotated tier — qt=44 is GemvMq4G256V2Prerotated, so a seed mismatch between pack_maple_head and ensure_mq_signs would produce wrong logits with no error at all. MEASURED, all four heads, same setup (gfx1151, KV bf16, 2048 teacher-forced tokens vs the bf16 reference, 3 paired interleaved reps for speed): head bpw mean KL top-1 decode mq4v2 4.25 0.0744 88.5% 165.8 tok/s mq4 5.00 0.0772 89.2% 161.8 tok/s q8 8.50 0.0511 91.9% 144.3 tok/s bf16 16.00 0.0511 91.7% 117.6 tok/s THE HYPOTHESIS FOR ADDING mq4v2 IS REFUTED. The prediction was that a separate fp16 scale/zero per 128-weight half would pull KL back toward q8's 0.0511 while keeping mq4-class throughput, which would have beaten q8 on both axes. It recovers only ~11% of that gap (0.0772 -> 0.0744) and top-1 actually drops below mq4 (88.5% vs 89.2%). Scale granularity is not the binding constraint here; 4-bit itself is. So q8 remains the default: mq4v2 buys +14.9% decode for +46% mean KL and -3.4pp top-1, the same poor trade this stack already rejected for mq4. mq4v2 IS however strictly better than mq4 on every axis — lower KL, faster, and 15% smaller (4.25 vs 5.0 bpw). Anyone wanting the fast head should use mq4v2; qt=30 now has no remaining advantage. Verified: coherent generation with clean EOS on the real checkpoint (the rotation contract holds), full workspace --all-targets build, and the complete --lib suite green (39 test binaries, 0 failures). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FdJ3XSghbyZZ9Ri2srFnm5
…ing qt=30
mq4v2 (qt=44) beats mq4 (qt=30) on EVERY axis, so qt=30 has no remaining
workload. Measured on gfx1151, KV bf16, 2048 teacher-forced tokens vs the bf16
reference, 3 paired interleaved reps for speed:
head bpw mean KL top-1 decode
mq4v2 4.25 0.0744 88.5% 165.8 tok/s
mq4 5.00 0.0772 89.2% 161.8 tok/s
Lower KL, faster, and 15% smaller. `--head-quant mq4` is therefore removed from
the CLI: it now errors with `[possible values: bf16, q8, mq4v2]`.
DEPRECATE THE PRODUCER, NOT THE READER. qt=30 `.hfq` files exist on disk, so
the quant_type -> DType arm for 30 STAYS. To keep those two things from drifting
apart, the mapping is extracted into the pure `maple_dtype_for_quant_type`, and
four tests pin the reader contract without needing a GPU — including
`deprecated_qt30_head_still_loads`, whose whole job is to fail if someone later
"cleans up" the deprecated carrier and silently breaks every existing model.
The map is append-only in practice and now says so.
q8 REMAINS THE DEFAULT. mq4v2 buys +14.9% decode over q8 for +46% mean KL and
-3.4pp top-1 — the same trade this stack already rejected for mq4. mq4v2 is the
right choice only when throughput dominates.
Verified end to end, both halves of the deprecation:
* the CLI rejects `--head-quant mq4` with the correct possible-values list
* the existing qt=30 model on disk still loads and generates coherently with
clean EOS at 151.0 tok/s
* full workspace --all-targets build; complete --lib suite green
(39 test binaries, 0 failures)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FdJ3XSghbyZZ9Ri2srFnm5
Two separate gaps between our 4-bit heads and the Q4_K head DeepGrove ship,
both measured on Maple's real lm_head (relative L2 vs the original bf16
weights, 3000 sampled rows):
4-bit, 1 scale per 256 (qt=30 class) 0.11814
4-bit, 2 scales per 256 (qt=44) 0.10569
4-bit, 4 scales per 256 0.09312
4-bit, 8 scales per 256 (Q4_K class) 0.08001
8-bit, 8 scales per 256 (q8) 0.00471
1. GRANULARITY. Our 4-bit carriers used 1-2 scales per 256 weights; Q4_K uses
8 (per-32 scale AND min, with 6-bit quantized meta). Fixed by adding
`--head-quant q4k`, which was nearly free: QuantType::Q4K = 4,
DType::Q4K, gemv_q4k.hip and quantize_q4k all already existed. Only the
packer arm, the CLI value and the arch's qt->DType map were missing.
Unrotated, so unlike qt=30/44 there is no FWHT seed contract to keep in sync.
2. ENCODER. `quantize_q4k` derived each sub-block scale by plain min/max
(`range/15`), which is not the error-minimising scale -- a single outlier
stretches the grid and every other weight pays. llama.cpp instead runs
`make_qkx2_quants`: search nstep candidate scales around the min/max one,
solve the weighted least-squares fit for (scale, min) at each, keep the
lowest-error candidate. Ported faithfully from ggml-quants.c:799 with Q4_K's
own parameters (nmax=15, rmin=-1.0, rdelta=0.1, nstep=20) and its importance
weights sqrt(mean(x^2)) + |x|.
Effect on the same tensor: 0.0799 -> 0.0720. For reference DeepGrove's
PUBLISHED Q4_K head measures 0.0731 against the same base weights; the small
remaining difference is the 6-bit super-block scale quantization this
measurement omits. The layout was already GGML-compatible -- only the
encoder was weaker. This improves EVERY Q4K tensor in hipfire, not just
this head.
WHILE VERIFYING THIS, TWO THINGS WERE SETTLED:
* DeepGrove did NOT post-train or specially calibrate their head. Their
published Q4_K `output.weight` sits at 0.0731 against the original bf16
lm_head -- exactly where quantizing those same weights lands -- with no
zeroed rows and no rescaling (max|w| 0.55708 vs 0.55859). It is a plain
quantization of the identical checkpoint.
* ROTATED Lloyd would be catastrophic for the BODY, confirming qt=51's `U`.
Maple's weights are exactly {-s, 0, +s}: measured on a real expert tensor,
every 256-block holds exactly 3 distinct values (min 3, max 3), so a 4-level
codebook is EXACT -- relative L2 0.000000. FWHT rotation mixes 256 weights
together, raising that to 17-37 distinct values per block and 2-bit Lloyd
error to 0.341803. Rotation helps dense distributions; it destroys this one.
q4k is offered, NOT defaulted: q8 is still 17x more accurate than any 4-bit
head, and on this stack the throughput gain does not pay for that.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FdJ3XSghbyZZ9Ri2srFnm5
…coded 128
`flash_partials` was sized with `max_seq.div_ceil(128)`, which is only correct
where the decode tile IS 128. `q8_flash_tile_size` returns **32 on gfx1100**
(RDNA3), so the decode kernel there computes 4x as many tiles as a
128-derived allocation assumes and indexes
`partials + (h * max_tiles + tile_id) * (2 + head_dim)` against them.
NOT A LIVE OVERFLOW, and the commit should not be read as fixing one. The
trailing FLASH_PREFILL_SUBBATCH (64) factor left enough slack to absorb the 4x:
max_seq arch tile decode needs alloc margin
32768 gfx1151 (RDNA3.5) 128 532,480 34,078,720 64.0x
32768 gfx1100 (RDNA3) 32 2,129,920 34,078,720 16.0x
What was wrong is the coupling, not the arithmetic. This was a FOURTH
independent copy of tile-size logic, consulting neither source of truth
(`q8_flash_tile_size` for decode, `attn_tile_size` for batched prefill), and
`launch_asym_flash_batched` already carries a comment about "the corruption bug
three independent copies of this exact logic caused". RDNA3 silently gave up
75% of its margin for a reason nothing in the code stated, and the next arch or
subbatch change could have taken the rest.
Deriving it also makes `HIPFIRE_Q8_FLASH_TILE` consistent: an operator override
now moves the allocation instead of quietly consuming the slack.
Batched prefill was already safe by construction and is untouched — it derives
`sub_batch` from the live buffer capacity, so a smaller tile shrinks the chunk
rather than overflowing.
Verified on gfx1151, where the resolved tile is unchanged at 128:
* mean KL 0.0511, bit-for-bit the same as before this change
* `HIPFIRE_Q8_FLASH_TILE=32` reproduces gfx1100's tile CHOICE locally —
mean KL 0.0493 and coherent generation, so the RDNA3 decode geometry is
exercised here rather than merely reasoned about
* full workspace --all-targets build; --lib suite green (39 binaries)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FdJ3XSghbyZZ9Ri2srFnm5
Shipping one head carrier per full model duplicates the identical 6.17 GB body
every time. Three variants cost 19.63 GB; a base plus two head overlays costs
7.30 GB, and switching heads becomes a 175-635 MB download instead of 6.5 GB.
`--head-only` writes a normal `.hfq` containing just `lm_head.weight` at the
requested `--head-quant`. Deliberately a NORMAL container with the same arch_id
and the same LOGICAL SHAPE as the base's head, because that is exactly what
`HfqFile::attach_overlay` already accepts: it shadows by name and permits the
quant tier to differ. Nothing in the overlay mechanism is relaxed.
A HEADLESS BODY IS DELIBERATELY NOT OFFERED. It would require letting an
overlay introduce names the base lacks — and that check ("tensor not present in
base — overlay likely built for a different model") is precisely what stops a
wrong-model overlay being spliced in silently. It would also ship an artifact
that cannot run alone. So the base keeps the recommended q8 head and is
runnable as-is; q4k and bf16 ride as overlays.
Validated against the SHIPPED base, both reproducing their monolithic builds
exactly:
configuration mean KL top-1
base q8, no overlay 0.0511 91.9%
base + q4k head overlay (188MB) 0.0640 90.1% (monolithic q4k: 0.0640)
base + bf16 head overlay (635MB) 0.0511 91.7% (monolithic bf16: 0.0511/91.7%)
Generation through an overlay is coherent. Build cost is 32 s versus ~10 min
for a full convert.
bf16 needs no special case: `convert_tensor` routes a bf16 head to a
`QuantType::BF16` passthrough and never reaches `pack_maple_head` (which has no
Bf16 arm). An earlier guard here claiming otherwise was wrong and is removed.
NOT YET PRODUCTISED: attaching an overlay currently goes through
`HIPFIRE_REAP_PLAN` pointing at a dir containing `overlay.hfq`, which is how
the validation above was run. A `--head` selector and registry `heads` entries
are the remaining work; the storage and correctness question this commit
answers is independent of that plumbing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FdJ3XSghbyZZ9Ri2srFnm5
Attaching a head overlay previously required HIPFIRE_REAP_PLAN pointing at a directory containing `overlay.hfq`, and the diagnostic said "reap: overlay ACTIVE" — wrong mechanism, wrong message for a head swap. Adds `HfqFile::attach_head_overlay`, `load_maple_from_hfq_with_head`, and `maple_coherence --head <head-only.hfq>`. Ordering is load-bearing and commented: the overlay attaches BEFORE `MapleWeights::load`, because the loader resolves `lm_head.weight` through the same `find_tensor_info` path the overlay shadows. Attaching afterwards would silently serve the BASE's head and hand back a model that looks correct and is not the one requested. Failure is an ERROR, not a warning. The REAP path warns and proceeds unpruned, which is right there — it fires on an env var that may belong to an unrelated model. A head overlay is requested explicitly, so falling back to the base head would be answering a different question than the one asked. A NEGATIVE CONTROL FOUND A REAL BUG. Passing a full model to `--head` "succeeded": every tensor name exists in the base at a matching shape, so attach_overlay's arch/name/shape guards all passed and the model silently shadowed itself — while printing all 18,651 tensor names, 918 KB of diagnostic. A head overlay must now contain ONLY `lm_head.weight`, and the listing is gone. It is refused with: head overlay "...": expected only `lm_head.weight`, found 18651 tensor(s) including `model.layers.0.input_layernorm.weight` — this looks like a full model, not a `hipfire-quantize --head-only` build Verified on the shipped base: q4k and bf16 overlays both attach and generate coherently with clean EOS; a full model is refused; full workspace --all-targets build; --lib suite green (39 binaries). Remaining for shipping: registry `heads` entries so `hipfire run` can fetch an overlay by name. The mechanism and its guards are done. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FdJ3XSghbyZZ9Ri2srFnm5
…models
Adds `heads: {name -> Sidecar}` to ModelEntry and populates it for
maple-preview with the two alternative carriers, each a single-tensor `.hfq`
from `hipfire-quantize --head-only`:
q4k maple-head-q4k.hfq 188 MB sha256 deff26e9...
bf16 maple-head-bf16.hfq 635 MB sha256 94cde3ad...
The BASE keeps the recommended q8 head and runs standalone; these only change
what a different carrier COSTS. Three full variants would be 19.63 GB and a
6.5 GB re-download to switch; base plus two overlays is 7.30 GB and a
188-635 MB download.
Validated against the shipped base — both reproduce their monolithic builds
exactly, so an overlay is not an approximation of a full build, it IS one:
base q8, no overlay 0.0511 KL 91.9% top-1
base + q4k overlay 0.0640 90.1% (monolithic: 0.0640)
base + bf16 overlay 0.0511 91.7% (monolithic: 0.0511/91.7%)
Heads are validated exactly like triattn/mtp/dspark by chaining them into the
same digest check. THAT CHAIN IS THE WHOLE POINT and is easy to omit: adding a
field to the struct makes it round-trip but does NOT make the validator look at
it, so a head with a malformed sha256 would parse and ship unverifiable.
`heads_sidecars_are_digest_validated` asserts the negative directly, and was
mutation-tested — with the `.chain(entry.heads.values())` removed it FAILS, and
with it restored it passes. It also carries a control proving the rejection is
about the digest rather than `heads` being unparseable.
Verified: full workspace --all-targets build; --lib suite green (39 binaries).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FdJ3XSghbyZZ9Ri2srFnm5
The indexed HFQ4-G256 gate/up kernel assumed K/256 was divisible by four and dropped Gemma4 K=2816 groups 8..10. Add fixed eleven-group variants so the tail is evaluated without compiler-generated private scratch, retain gfx1100 cache-policy selection, and register the new symbols with replay metadata. Validated with Gemma4-26B-A4B artifact sha256 45da530b43e0e8ea2cffd77fe17c69bfb8dce3ec4dde2a26a9b68ad428d01f1c on gfx1100 and gfx1201: stable 1263-launch captures, HIP/PM4 exact shadow parity, and byte-identical greedy E2E output for prompt md5 43ca0d15712d3dfb777b51ae76d8fd5f. (cherry picked from commit 2038b9f)
(cherry picked from commit dea9a75)
(cherry picked from commit 689ca8e)
Only hold AwaitingThought while the decoded prefix can still become the canonical thought header. Otherwise route the stream as answer content and strip only an orphan channel control token, preserving its payload. Also narrow marker-prefix detection so marker-plus-content is never discarded as a unit, and add chunk-boundary and EOS regression coverage. Fixes #678 (cherry picked from commit 97cc799)
(cherry picked from commit 21d67fb)
(cherry picked from commit fe4d128)
Keep conservative dependency acquires on gfx12 and use a system-scoped PM4-IB vendor packet boundary so retained VMEM producers are visible to their consumers. Extend the Gemma4 shadow oracle across logits, sliding/full KV, recurrent state, scratch, and captured blobs, and retain the indexed HFQ4-G128 MoE down path used by the validated lowered graph. (cherry picked from commit 3bd12c1)
Let the Gemma-specific evaluation harness request thinking_enabled=false so the model template emits its native empty thought channel. Keep --closed-think as a compatibility alias and record any unexpected reasoning event as a no-think violation. (cherry picked from commit 17abd04)
(cherry picked from commit 7b4e93f)
(cherry picked from commit e08991a)
The registry already declared maple's `heads`, and the loader could already
attach one, but nothing joined them: `hipfire run` had no way to ask for a head
variant. This threads it CLI -> params -> daemon -> LoadCtx -> maple carrier.
hipfire run maple-preview --head q4k "..."
hipfire run <model.hfq> --head <head-only.hfq> "..."
`--head` takes a REGISTRY NAME or a PATH. The path form is not a convenience:
loading a model by path has no registry entry, so a name cannot resolve there
and only a path can work.
Unknown names REFUSE and list what exists, rather than falling back to the
model's own head — a silent fall-back would serve a different model than the
operator asked for, and the whole point of the flag is choosing the head:
--head nope: not a file, and this model has no such head variant
(available: bf16, q4k)
A declared-but-missing overlay refuses too, naming the path it looked for.
Verified end to end on the real model, through the daemon and carrier:
* `--head <path>` and `--head q4k` / `--head bf16` by registry name all
attach ("head overlay: 1 tensor(s) ... shadow the base") and generate
* unknown name lists `available: bf16, q4k`
* loading by path with a name errors correctly (no registry to resolve it)
* full workspace --all-targets build; --lib suite green (39 binaries)
FOUND WHILE TESTING, AND IT AFFECTS MORE THAN THIS FLAG: the CLI reads the
registry from DEFAULT_REGISTRY_URL
(raw.githubusercontent.com/warpfront/hipfire/master/registry/v1.json) and
caches it for 24h in ~/.hipfire/registry.cache.json. Editing registry/v1.json
on a branch changes NOTHING for a running client until it lands on master. The
local cache here still had `heads: {}`, `default_kv_mode: null` and
`sampling.temperature: 0.6` — so the bf16 KV default and the vendor
temperature from earlier in this branch are also inert until merge. Testing
against a branch needs HIPFIRE_REGISTRY_URL=file://.../registry/v1.json.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FdJ3XSghbyZZ9Ri2srFnm5
…urce Three problems, one root cause: I had been editing the wrong file, and the result was invisible anyway. 1. I HAND-EDITED A GENERATED FILE. `registry/v1.json` says "GENERATED by scripts/registry_gen.py — do not hand-edit. Edit registry/models.json". My earlier bf16 default, temperature 1.0 and heads went into v1.json only, so the next generator run would have silently reverted all of them. They now live in registry/models.json and v1.json is regenerated from it. The regeneration is worth more than tidiness: the generator PROBES Hugging Face and derives sha256/size_bytes itself. The head digests it produced match the uploaded files exactly, so the registry cannot drift from what is published — where a hand-copied hash could. 2. THE GENERATOR REJECTED bf16. It carries its own KNOWN_KV_MODES allowlist — a third copy alongside hipfire-config's KV_MODES and kv_mode.rs's per-site policies — and failed closed on `default_kv_mode: bf16`. Added, with a note pointing at the other two. It also had no notion of `heads`, so they are now annotated per entry like triattn/mtp (a map rather than a single sidecar). 3. BRANCH REGISTRY EDITS WERE INERT. `load()` resolves cache -> network(master) -> stale cache -> bundled, so a locally built binary — whose bundled registry IS its branch's — was silently overridden by a 24h cache or a master fetch, with nothing reporting which source won. That cost a real debugging detour: a branch's `heads` map read as empty and looked like a code bug. Now the bundled registry wins when its `generated_at` is NEWER. No new configuration: `generated_at` already exists, the generator stamps it on every run, and its %Y-%m-%dT%H:%M:%SZ form compares correctly as a string. The override is reported through the existing warnings channel rather than happening silently. This does NOT freeze clients at their build-time registry. A released binary's bundled copy is older than master's by construction, so the fetch still wins and users keep getting new models without upgrading — pinned by `older_bundled_registry_defers_to_the_fetch`, the direction that would otherwise break distribution. `equal_timestamps_keep_the_fetched_registry` stops the two sources flapping. Verified with NO env vars and the real Aug-31 master cache in place: `hipfire run maple-preview --head nope` now reports "available: bf16, q4k" from the branch. Previously it said the model published none. Full workspace --all-targets build; --lib suite green (39 binaries). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FdJ3XSghbyZZ9Ri2srFnm5
…per_var + maps) Rebase resolution for PR667 onto beta 0db445e, stacked on PR734. - Delete LOWERED_GENERATE_REFUSAL / gemma4_lowered_refusal gate: the complete enabled path (generate_gemma4_lowered dispatch, max_seq scratch/KV authority, HFQ4-G128 row tails, Q8 ring rollover, K2816 tail specialization) is present, so admission no longer refuses. - Migrate 667-new HIPFIRE_JINJA_CHAT read to hipfire_config::developer_var. - Regenerate touched crate maps from beta.
Signed-off-by: alpineq <alpineq@protonmail.com>
# Conflicts: # crates/hipfire-generate/tests/qwen35_reset_hw.rs # crates/hipfire-loader/src/admission.rs
Copy-only EP row transport on the sealed path: Gpus::gather_slots_to_root_f32 (multi_gpu.rs:1900) and Gpus::broadcast_root_row_f32 (multi_gpu.rs:1952), with pure preflights check_ep_gather_slots (:1990) and check_ep_broadcast_row (:2052). Uses only boundary_copy (:471) plus wait_boundary (:537) over DeviceBuffer::byte_view spans; adjacent same-owner slots move as one span; root-owned slots skip the self-copy; broadcast overwrites (never adds). All ranks, owners, checked byte counts, and capacities preflight BEFORE the first copy. No arithmetic on row values; no peer-scratch allocation (destination rows already exist). Owners come from the caller-provided sealed mapping; no stride logic in this file. Gates (paste of measured output): cargo test -p hipfire-runtime --locked --lib multi_gpu -- --test-threads=1 test result: ok. 16 passed; 0 failed; 1 ignored; 0 measured; 677 filtered out (incl. ep_gather_spans_group_adjacent_same_owner_ep2/ep4, ep_gather_spans_degenerate_groupings, ep_gather_n1_plans_root_only_spans, ep_gather_rejects_bad_owner_rank_and_capacity, ep_transport_checked_byte_math) HIP_VISIBLE_DEVICES=0,1 flock -w 3600 /tmp/hipfire-gpu.lock cargo test -p hipfire-runtime --locked --lib multi_gpu -- --ignored --exact --test-threads=1 multi_gpu::tests::ep_slot_row_transport_2gpu test multi_gpu::tests::ep_slot_row_transport_2gpu ... ok test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 693 filtered out cargo build --release 2>&1 | tail -2 Finished `release` profile [optimized] target(s) in 13.21s rustfmt --edition 2021 --check crates/hipfire-runtime/src/multi_gpu.rs -> clean (RUSTFMT_CLEAN) The 2-GPU test proves transport only, not model parity. No push.
Wave 1 slice C1: plan-bound compact residency for sealed EP owners. Rank-aware adaptation (crates/hipfire-runtime/src/sealed_moe.rs:1400): adapt_expert_execution_plan(plan, local_rank) selects rank_ownership()[local_rank] and verifies logical rank, global_to_local on all ranks, mesh devices, physical device, and compact local slots before publishing any table. Every caller migrated (plan_single_expert_execution passes 0 at :1701; in-file tests pass explicit ranks). No compatibility wrapper kept. Deterministic execution contract/fingerprint (runtime :1613, :254; dispatch pipeline/sealed_moe.rs:497,547): ExpertExecutionContract carries group/layer, source fingerprint, mesh epoch, physical rank list, parallelism, assignment, per-expert owner/slot map, execution string, and ordered collective rows. fingerprint() renders one canonical string (pointer-free, comparable across ranks/processes); is_canonical_ep() admits only EP + indexed-decode-slot-order + a nonempty all-EP-allreduce schedule. ExpertTable::with_execution_ contract(:771) cross-checks owner/slot per expert at attach; BoundMoeExperts::execution_contract(:1260) exposes it read-only. Single->EP repartition (runtime :1733): accepts only sealed Single plans, rebuilds manifest/spec/sources from attested record metadata (packed extents recovered as per-expert bytes x n_experts; fused carriers, sidecars, aliases carried verbatim), and reruns plan_manifest + plan_expert_execution. No e % N tables are hand-built; collective rows come from the planner. Local experts ordered by local_slot (dispatch :991-1004): the cache sorts owned records by sealed slot instead of global id, still requiring compact 0..k. Compact live bind (dispatch :1123, builders :2221): bind_live_compact sits alongside bind_live (Single API untouched) and proves owned entries point at plan-mapped local tensors in slot order, non-owned entries point at owned layout-compatible zero dummies, AWQ/tag presence/capacity match, GPU tables have exact identity/capacity, and the device matches. The mapping fingerprint is retained (:1172) with the tensor identities. Qwen refusals untouched. CPU mesh fold (runtime :2224, helpers :2185,:2193): owner ranks materialize raw rows; one flat slot-order fold accumulates w[i]*v[i]; rank count is absent from the arithmetic. The even/odd partial association survives only as a tests-only legacy helper. Production ownership still derives only from plan_expert_execution; e % N appears solely inside test oracles restating the planner's own Stride/Contiguous policy definitions, and in the planner itself. Measured gates (no GPU; oracles untouched): test result: ok. 20 passed; 0 failed; 0 ignored; 0 measured; 681 filtered out; finished in 0.01s test result: ok. 21 passed; 0 failed; 0 ignored; 0 measured; 242 filtered out; finished in 0.00s Finished `release` profile [optimized] target(s) in 14.04s rustfmt --edition 2021 --check clean on both files (no diff). No push. No new crates. No ratchet change. bypass_slack untouched.
Sealed streaming load_weights_ep_rank(hfq, gpu, config, mesh, physical_devices, shard, rank): validates mesh/rank/device identity, assignment, and source/config seals before a private RAII EP load context; loads only rank_ownership[rank].global_expert_ids in local-slot order; builds pointer tables from global_to_local with layout-specific zero dummies for non-owned entries; publishes plan, tables, compact live cache (bind_live_compact), dtype tables, dummies, and local experts atomically. Qwen35EpShardInfo.expert_to_rank derives from the sealed plan, never e % N.
build_expert_binding takes ExpertBindingTarget::{Single, ExpertParallel}; EP emits ExpertSharded policies and execution indexed-decode-slot-order. shard_moe_experts/shard_all_moe_layers are two-phase (prepare every layer against the untouched owner, then infallible swap; displaced tensors retired until teardown). Public set_ep_expert_shard/EpShardGuard removed; oracle and ep_decode_parity moved to Gpus::init_ep + load_weights_ep_rank with oracle assertions byte-identical. Daemon qwen35_ep_moe_refusal unchanged.
Measured on gfx1201 (maintainer run): sealed_ep_tests 8/8 -- compact bind proof, source-scan/owned-upload/dummy-alloc/pointer-upload/compact-bind fault rollbacks, later-layer prepare fault leaves old owner, ornith EP2 rank-0 load smoke (fixture sha256 84103fcc...). Lib suite 195 passed. bypass_slack 0. Test-only serialization lock uses std::sync::Mutex (no new dependency).
Preserve the canonical root-route correctness baseline before efficient EP cutover. Root-authoritative slots remove independent routing assumptions; destination-owned peer copies enforce producer and source-reuse dependencies. Preflight routes before mutation and drain queued-copy error paths. R9700 gfx1201 Ornith MQ4R sha256 84103fcc8ade42aa2ac8ec01176df7a4ead5e94810597c9fae2f6763152a3ac6: EP2/Single and EP4/EP2 each 27 positions max logit difference 0, fwht3 KV/Q8 state, NINEPATH=0, launch blocking off, diagnostics removed. 25 seal tests and active-stream two-GPU source-overwrite regression pass. TP4 contiguous-Q8 registry-sampled battery 5/5 inspected; explicit reset, abort, reuse, unload/reload smoke passes. VMM remains refused; no full G5 or performance acceptance claim.
Replace canonical slot transport with root-authoritative GPU top-k routes and fixed-rank partial reductions. Reuse route/reduction events, enable peer access after staging, batch production prefill, and remove repeated allocation from sealed expert validation without caching mutable owners. Wire continuous-batch and sequential fallback to the same owned reduction lease. Embed independent KV kernel headers in JIT source/cache identity. Reject unusable batch staging and defer prior-resident retirement until all staging succeeds; invalidate graphs before freeing EP resources. Verification on 4x R9700/gfx1201: - Release daemon/CLI build and 29 sealed-MoE tests pass. - Three physical GPU transport/reuse tests pass. - EP2 and EP4 snapshot oracles pass 27 positions of within-mesh route-byte and post-reduction residual agreement; cross-route logits are diagnostic, not an equivalence gate (worst Single/EP2 1.228, EP2/EP4 0.001897). - Native daemon batch lanes, sequential fallback, cancellation/reuse, reset, reload and successful replacement pass with decoded text inspected. - Injected post-free staging failure rejects the new load without a loaded ack; previous resident still answers. Injection removed before normal build. Exploratory short probe only: one fresh process, 3 warmups + 5 requests, TP4, q8, speculation off, greedy, max_seq=4096, max_tokens=128, prefill chunk override=2048. Median 2135.2 prefill and 65.5 decode tok/s. Not a three-fresh-process performance claim and not the hundreds-of-tok/s target. Model: ornith-1.5-35b-a3b.mq4r, md5 33e4f6bc2e48a527fbfbf8a101ef44b0, sha256 84103fcc8ade42aa2ac8ec01176df7a4ead5e94810597c9fae2f6763152a3ac6. Prompt: benchmarks/prompts/glimmer_prefill_1024.txt, md5 0ee8f86ada3683eda452bc294ec824a9, 1131 tokens. Measured daemon md5: 4a159e47d25392aa762ed2dc1780978a. Measured CLI md5: 7d43ef7d927042071ae58e13fc0c733d. Full session_coding.json acceptance, request-scoped prepared admission, and EP replay remain paused; no PM4 admission or full G5 acceptance claim.
…ding The route was excluded under HIP graph capture but not under Redline retained/PM4 recording, which would freeze the long-context tile geometry into a replay. Gate on gpu.replay.is_recording() at both admission sites; bench_flash_rows gains a --check exit for the oracle.
# Conflicts: # crates/hipfire-arch-qwen35/map.md
The 755 merge added two A3B sealed-MoE call sites against the pre-merge 4-arg signatures; beta's assert_done_stop/clean_bytes take expect_dflash. A3B has no draft, so false, matching the sibling A3B round.
# Conflicts: # crates/hipfire-arch-qwen35/map.md # crates/hipfire-runtime/map.md
…via developer_var
Replace the raw std::env::var("HIPFIRE_TOOL_GRAMMAR") read with
hipfire_config::developer_var("HIPFIRE_QWEN35_GRAMMAR") so the slot
backend honours the same documented kill-switch (=0 disables) as the
non-slot Qwen path (hipfire-generate/src/qwen.rs, docs/env-vars.md).
No new key.
…g history The slot backend and the non-slot daemon both publish arch qwen3_5 / qwen3_5_moe (slots.rs cpu_preflight, daemon main.rs arch match), which include_reasoning_content did not match, so rich tool history dropped reasoning_content. Accept the underscore spellings at the recogniser; add a unit test covering qwen3_5, qwen3_5_moe, and a negative.
Qwen35MtpDrafter::new read HIPFIRE_DFLASH_CKPT_RESUME / HIPFIRE_CACHE_CKPT_INTERVAL / HIPFIRE_CACHE_CKPT_MAX inline and mtp_repair_terminal_prefix re-read HIPFIRE_SPEC_WINDOW_ROLLBACK per repair. Policy now resolves once via hipfire_config::mtp_cache_policy() (+ mtp_ngram_enabled / ngram_mod_triple for the pre-existing NGRAM reads); the drafter stores resolved fields, mirroring build_dflash_speculator's one-shot resolution. No developer_var/std::env reads remain in mtp_speculator.rs.
…int rewind New ignored HW test mtp_cache_byte_identity (+ minimal mtp_live_state attestation hook): compares target DeltaNet bytes, retained-prefix trunk/MTP KV bytes, prev_hidden bytes and token ids of the repaired/rewound session against a cold render, plus an identical greedy continuation window. Currently FAILS on repair dn_s (see report).
Separate mtp_step_oracle binary (arm D stepped-no-repair + cold B2 of D's full stream; two sessions fit where three OOM). free_session gains the daemon's unload hooks (invalidate_weight_caches/graph_state) after a third-sequential-load OOM. Main binary keeps A-vs-B first-diff row / DN-layer diagnostics.
…, slot tool turns (#753)
…or; env-var source row; AGENTS date
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
v0.3.1 — beta → master
Maintainer-integrated candidate:
07d08aa72d1f114dd5823948c289b695173437cc.Keep draft pending an approving review from another account; current CI passes. Master is unchanged; no protection bypass is authorized.
The Gemma boundary/rollback correction wave landed in
bfa1c906e; rich Qwen history-token preservation landed in5773f6249; #741 and #746 (gfx1100 verifier multi-row KV scan, MQ4V2 residual scratch removal) were folded in at07d08aa72with a fresh gfx1100 validation cycle below. Previous review findings are addressed, but no second independent review verdict is claimed. Workspace/Cargo.lock and citation metadata are 0.3.1. This PR does not tag or publish a release.Latest candidate receipts — supersede earlier final identities below
session_coding.jsoncompleted all eight turns normally. All seven follow-ups reused the complete prior token prefix (6,347 through 35,541 cached tokens); both recall checks passed, no empty/runaway/attractor flags. Actual reasoning was enabled: the legacy harness--thinking offbudget was ignored by the effort-native contract. This is not a no-thinking claim.LCP == prior_len < rendered_lenhits. The none/cap1 control emitted zero thinking words. These are cache proofs, not full session-quality passes: both runs failed the literaldeduperetrieval check; the capped reasoning run additionally hit the existing post-latch answer limit. Those failures are retained, not waived or relabeled._pr_smokes/release031-cache-gemma-redline.jsonispass:true.3250df338276d757e5f18b4d59fc6ab2. Source was built before the final comments/maps/changelog-only cleanup and commit stamp. Reports:_pr_smokes/release031-fwht3-session.json,release031-ar-rich-session.json,release031-ar-nothink-session.json, and matching serve logs.5773f6249). Earlierbfa1c906erequired CI passed; advisory Clippy reported the existing approximate-PI literal intest_dflash_hidden_scatter_gfx1100.rs:29.Folded gfx1100 perf PRs — #741 (alpineQ) and #746 (HUSRCF)
Both PRs merged onto beta unmodified (only the generated
crates/rdna-compute/map.mdconflicted and was regenerated). Validated on the RX 7900 XTX (gfx1100, hipx) against the unfolded5773f6249build, canonical Qwen3.8-27B MQ4XT (sha2569f91556f…) + MQ4V2 draft, Q8 KV, fresh process per sample, arm order alternated,HIPFIRE_DFLASH_CTX_CAP=0,HIPFIRE_VERIFY_GRAPH=0. Candidate daemon MD56db9ea54cb65b274ed8ea8484d3b2810; baselinea31178c687f5d6cd756ce17dec84f8b1.df5dedc8), DFlashb4d0b63c, #741 fixture), DFlashDFlash fired in 12/12 speculative samples (τ 10.55 short / 1.95 long, identical across arms). At 21.5K on this box the fold moves DFlash from below AR (36.6 vs 40.1) to above it (48.6 vs 41.2) — a partial answer to #693 on gfx1100 at this fixture, not a general claim. The long/DFlash r1 pair ran cold on both arms (~31); r2/r3 are the evidence.
Correctness on the candidate:
serve_harnessbattery 5/5 and chain 5/5, all normal stop, no empty/attractor/runaway, chain prefix cache growing (0/284/493/674/939 cached), decoded text inspected. Redline on gfx1100: stable prefill (1124 launches, 21 kernels) and decode (659 launches, 16 kernels incl.gemv_mq4g256v2_residual), PM4/HIP/blob bit-exact across 3 positions,pass: true.hipfire-arch-qwen35lib tests 193 pass.Known pre-existing flake, not introduced here:
rdna-compute dispatch::tests::upload_raw_copy_failure_hip_frees_ownerfails ~50% under default test parallelism on a GPU host (free-VRAM equality race) on both baseline and candidate; deterministic pass with--test-threads=1. Not a CI job (no-GPU). Follow-up issue to file.The sections below retain earlier, fixture-bound integration evidence; older binary IDs and candidate hashes are historical, not the current candidate. Current daemon-line ratchet is 4155 → 4411 (declared), not4410.
Scope
c91f4b65is not part of perf(gfx11): optimize Qwen DFlash speculative verify #695 or this promotion.Gemma maintainer repair and proof
Prerequisite #734 lands before #667. Single-token and separate batched-prefill APIs share indexed MoE semantics; Gemma Q8 batched projections explicitly retain F32 math. The HD512 batched-attention Q preload and four-term dot association now match decode. No change to the other head dimensions, descriptor/window ABI, or final reduction. Temporary divergence probes were removed.
Fixture:
gemma-4-26b-a4b-it.hfq4g128-maintainer.hf4, 15,343,188,028 bytes, SHA25611cf46cba97f5e279d351f9d31cf4bdd78cb1fbc7c16da2433e6141ae7e07d53. Built from the locally available Google HF snapshot; this is not the author's unavailable artifact and is not a new registry publication.On gfx1201 / HIP 7.15:
Limits are explicit: eight-token serial harness capture exceeds the existing 4096-launch recorder budget and is not admitted by this report. The raw124-token parity prompt produces the same
[]loop in both arms—numerical parity is not coherence. Gemma serving statistics/finish metadata are absent in these reports; no token-rate claim is made. No new gfx1100/gfx1151 Gemma proof, broad quality admission, or performance claim.Combined product receipts
Canonical Qwen3.8-27B XT target SHA256
9f91556f7e0431a077d03756a7102d0154108757289e6e5fe9a2d204c0c9eeb7; draft SHA256d0a74a232a0e2166d889f823e91e0fbf778d21dd9668d7de055cdecb065401bc(both rehashed). DFlash battery and chain each5/5, all decoded answers inspected, all DFlash-active and normal stop; chain reaches998 context tokens with cached prefixes. Q8, greedy, thinking off, max_seq4096, max_tokens512. One cold-load warmup exceeded the harness's180s wait; retry completed, and the chain explicitly uses600s. No loading error or performance claim.Earlier per-PR receipts remain scoped to their tested artifacts: #736 six real GPU rollback tests; #734 four K2816/K2048×MQ4/HFQ numerical cases; #670 head/KV native matrix, invalid-head retention and window checks; #704 exact19-position manifest/legacy parity, real AWQ scaling and four-cycle post-warmup VRAM plateau. The tiny #704 fixture's stock512-token battery fails identically on unchanged beta and candidate; only its coherent scoped smoke is claimed. Maple does not support the Redline shadow route; no Maple PM4 proof is claimed.
Final unit validation exposed a synthetic Llama fixture disappearing before its legacy reopen. Tests now retain uniquely owned NamedTempFile fixtures through every read; no test serialization, ignored oracle, or weakened assertion. The test-only override is
G3_FIXTURE, outside the productionHIPFIRE_*configuration namespace.Ratchets and limitations
Declared
RATCHET-RAISE: daemon_lines 4155 -> 4410andRATCHET-RAISE: bypass_total 237 -> 257; the PR retainsratchet-raise. Leanup21/21, crate maps43/43, environment ownership and ratchet-diff against master pass.Windows-host proof for #737 was explicitly waived by the maintainer. Linux/gfx1201 GPU roundtrip passed; Windows execution is not claimed. #694 remains optional WIP, open and outside this promotion.
Disposition
After promotion, close superseded #682 #686 #687 #688 #689 #690 #691 #692 #695 #700 #701 #708 #725 #726 #728 #729 #680 #697. Preserve their branches and attribution. #667 is closed after its attributed beta integration; #734/#735/#736/#670/#704 and earlier superseded PRs already have integration receipts.
Final local gates and identities
cargo build --release --workspace --all-targets --locked: pass.cargo test --lib --workspace --locked --quiet: pass on b8092f7, including repaired concurrent Llama fixture ownership.00c1ca17422f3073e9a0f92b88d83bee(hipfire 0.3.1, b8092f7).90abfc926a93d9b3048ac5838a3eb122: byte-identical to the Gemma and canonical DFlash runtime receipts. Serving CLI at b5 was334296da4b4b0d271ceb13a718146d4e; intervening changes are test-only fixture ownership/dependency/maps._pr_smokes/release031-{gemma-battery,gemma-chain,gemma-redline2,final-gemma,final-qwen-battery,final-qwen-chain}.json; no benchmark rate is promoted.