Add exact fail-closed Qwen4Exp GGUF support - #656
Merged
Conversation
Performance Comparison
|
This was referenced Aug 26, 2026
justinchuby
force-pushed
the
justinchuby-add-qwen4exp-gguf
branch
from
August 26, 2026 18:22
c1b734e to
1c09bee
Compare
justinchuby
added a commit
that referenced
this pull request
Aug 27, 2026
## Summary - classify every unresolved authoritative GGUF architecture, projector, tokenizer, MTP, and speculative-draft route exactly once - generate concrete implementation, evidence, ABI/dependency, artifact, and rejection batches from live registries - preserve exact registry reasons in machine-readable census records and reconcile PRs #645, #651, #652, and #656 without claiming open work as supported - distinguish ordinary runtime evidence from ORT GenAI heterogeneous-state schema issue #605 - pin availability-only F16/Q8_0 sidecar identities for LFM2-VL and Pixtral without implying tensor closure, graph parity, or runtime support ## Census - 285 unresolved routes: 144 architectures, 60 projectors, 57 tokenizers, 22 MTP routes, and 2 draft routes - 67 immediately implementable - 95 evidence-only - 100 dependency/runtime-ABI blocked - 4 artifact unavailable - 19 intentionally rejected ## Validation - generated documentation check - 1,371 focused GGUF tests passed, 1 skipped - broad deterministic suite: 8,467 passed, 62 skipped, 1 subtest passed - Ruff/format lint passed - strict mypy passed for changed source modules - independent incremental review found no significant issues ## Waivers - no large model payloads downloaded; availability pins use immutable repository revisions plus API-reported LFS SHA-256 and byte size - open PRs #651/#652/#656 are dependency records only and are not promoted as current-main capability - no unsupported cache/state ABI is forced; packed quantized support remains governed by existing fail-closed capability records Do not merge yet. --------- Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com> Copilot-Session: d1a98adf-88cb-471b-8c46-7a84083db24c
justinchuby
force-pushed
the
justinchuby-add-qwen4exp-gguf
branch
from
August 27, 2026 05:27
1c09bee to
f049303
Compare
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
justinchuby
force-pushed
the
justinchuby-add-qwen4exp-gguf
branch
from
August 27, 2026 05:32
f049303 to
5c0451b
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends Mobius’ GGUF integration to recognize general.architecture=qwen4exp as the existing qwen4_exp_text model type, while enforcing an explicit fail-closed policy that only supports exact header/config/tensor-contract preflight (no executable payload import) for the pinned Unsloth 3-shard artifact.
Changes:
- Add preflight-only GGUF architecture support for
qwen4expwith exact pinned metadata/tensor closure + explicit Hub identity gates that refuse payload download/materialization. - Expand sharded GGUF handling: stricter shard-set validation, deterministic merged metadata, shard identity paths for stable hashing across HF snapshot symlinks, and Hub resolution that downloads complete shard sets (with free-space preflight).
- Add Qwen4Exp-specific GGUF tensor-name mapping and weight preprocessing to fuse GGUF’s split indexer Q/K rows into HF’s fused
index_qk_proj.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/mobius/models/qwen4_exp.py | Fuse split GGUF indexer Q/K projection weights into HF’s fused index_qk_proj during weight preprocessing. |
| src/mobius/models/qwen4_exp_test.py | Add unit coverage for indexer fusion, preservation of official fused weights, and fail-closed validation. |
| src/mobius/integrations/gguf/_upstream.py | Extend the pinned llama.cpp census with an explicit post-census qwen4exp architecture entry. |
| src/mobius/integrations/gguf/_upstream_gap_test.py | Update census closure expectations to include the additional pinned architecture. |
| src/mobius/integrations/gguf/_tensor_mapping.py | Introduce Qwen4Exp GGUF→HF tensor mapping table and allow mapping for preflight-only specs. |
| src/mobius/integrations/gguf/_spec.py | Add preflight_only to the architecture spec contract and enforce internal consistency. |
| src/mobius/integrations/gguf/_shard_set.py | Strengthen local shard discovery/validation, add metadata merging, and expose identity paths + shard tensor source helpers. |
| src/mobius/integrations/gguf/_shard_set_test.py | Add extensive tests for shard discovery, metadata authority rules, Hub shard resolution boundaries, and integrity checks. |
| src/mobius/integrations/gguf/_runtime_evidence.py | Hash sharded artifacts using stable regular-file “identity paths” when available (e.g., HF blob store). |
| src/mobius/integrations/gguf/_runtime_evidence_test.py | Validate identity hashing behavior across snapshot symlinks via identity-path aliases. |
| src/mobius/integrations/gguf/_reader.py | Add _tensor_source() for per-tensor file/offset/record access (used by Tencent Q1_0 parsing). |
| src/mobius/integrations/gguf/_qwen4_exp.py | Add Qwen4Exp pinned Hub identity + exact header/config/tensor contract validation and deliberate payload fail-closed blocker. |
| src/mobius/integrations/gguf/_qwen4_exp_test.py | Add tests for exact config mapping, tensor mapping, manifest digesting, and strict Hub/payload gates. |
| src/mobius/integrations/gguf/_header.py | Add bounded header parsing that also extracts split bookkeeping into GGUFHeaderInfo with clearer truncation errors. |
| src/mobius/integrations/gguf/_docs.py | Document preflight-only routes and clarify split-set discovery and Hub split-set behavior. |
| src/mobius/integrations/gguf/_docs_test.py | Update generated census counts and ensure doc output matches new totals. |
| src/mobius/integrations/gguf/_config_mapping.py | Add Qwen4Exp GGUF→config extraction and postprocessing to construct the exact Qwen4ExpConfig. |
| src/mobius/integrations/gguf/_builder.py | Add full Hub split-set selection/download logic, strict bounded-header preflight paths, and Qwen4Exp fail-closed policy enforcement. |
| src/mobius/integrations/gguf/_builder_test.py | Add tests for sharded quantized builds, bounded-header edge cases, and preflight/download boundary behavior. |
| src/mobius/integrations/gguf/_arch_registry.py | Register qwen4exp as preflight-only (config + tensor-map supported; graph/runtime deferred; quantized import rejected). |
| src/mobius/integrations/gguf/_arch_registry_test.py | Adjust registry invariants to allow preflight-only specs (mapping/config without graph support). |
| src/mobius/integrations/gguf/init.py | Update public usage examples to show local shard paths and Hub shard references. |
| docs/design/qwen4-exp-text-core.md | Document the pinned GGUF artifact identity and explain the explicit payload fail-closed rationale. |
| docs/api/build_from_gguf.md | Update census totals and document split-set handling and the qwen4exp preflight-only route. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
justinchuby
force-pushed
the
justinchuby-add-qwen4exp-gguf
branch
from
August 27, 2026 05:48
e2e2f29 to
f1a5be7
Compare
Assemble strict local shard sets as one lazy logical GGUF model and resolve Hub sets at an immutable revision with cache-aware capacity preflight. Preserve quantized tensor paths, deterministic metadata, runtime identity hashing, and single-file behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Map the pinned qwen4exp configuration and complete 1,224-tensor header closure, including split QSA index projections and the metadata-only first shard. Reject the published mixed IQ1_S/IQ4_NL payload before download because neither native preservation nor full dequantization has a truthful bounded runtime path. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Pin every tensor name, logical shape, and dynamic qtype through the complete header-manifest digest, preserve official fused HF indexer weights, and reject unpinned config metadata. Make Hub header transport and truncation failures stop before payload download so architecture policy cannot be bypassed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Carry immutable Hub LFS manifests through resolution into reader checksum verification, enumerate renamed split siblings from bounded header metadata before download, and make primary-shard semantic metadata authoritative. Mark Qwen4Exp as header/config/tensor preflight-only until an executable ABI exists while preserving generic Hub fallback behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Separate split-header parsing from architecture dispatch so unrelated projector files do not poison renamed shard discovery, cap candidate range reads by the declared split count, and reject ambiguous directories mixing split and standalone GGUF models. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Reject oversized shard counts before allocation or network probing and enumerate local GGUF suffixes case-insensitively so discovery behaves consistently on Linux and case-insensitive filesystems. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Regenerate the support matrix on top of the authoritative remaining-route census and compress the shard-import overview without dropping its validation contract. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Keep immutable source pins in the design documentation and remove duplicate runtime constants that were not part of validation or dispatch. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Teach direct reader calls to inherit resolved shard paths and immutable Hub manifests, and distinguish bounded-header fallback so renamed ambiguous sets reject before a potentially partial download. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Regenerate the support matrix after the latest tokenizer-evidence census and keep the usage guidance below the documented concision limit. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Route text-side multimodal GGUF opens through the shard-aware reader and classify Qwen4Exp preflight work as runtime-ABI blocked in the generated remaining-route census. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Pin the exact float32 GGUF representation of the RMS epsilon so the real artifact header passes strict metadata validation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
justinchuby
force-pushed
the
justinchuby-add-qwen4exp-gguf
branch
from
August 27, 2026 05:58
f1a5be7 to
fbc3370
Compare
justinchuby
added a commit
that referenced
this pull request
Aug 27, 2026
## Summary - add the Qwen3.8 Flash-Next/Qwen4Exp three-model image-language package (`decoder`, `vision_encoder`, `embedding`) - reuse the source-identical no-DeepStack Qwen3/Qwen3.5 vision tower with a 2560-wide merger - preserve independent lexical `ple_input_ids`, fused `inputs_embeds`, four-channel text + M-RoPE positions, and every heterogeneous recurrent/KV/QSA/PLE state - cast processor float32 patches once at the BF16 vision boundary - emit explicit `mobius.state_manifest` and `mobius.unsupported_token_ids` contracts for direct ONNX Runtime orchestration ## Bounded multimodal streaming Both `qwen4_exp` and `qwen4_exp_text` use the bounded safetensors path; the eager `_download_weights` route is unreachable. One shard index validates config-derived deterministic PLE buffers and plans decoder, embedding, and vision bindings. Each graph is cloned, lazily bound, and initializer-folded in isolation; the package is updated only after every staged component succeeds. Missing tensors, corrupt PLE hash buffers, or fold failures leave the original package untouched. Parameter payloads remain `LazyTensor`s and no source state dict is retained. The approximately 95 GiB PLE table is allocated once only when materialized/serialized and populated one checkpoint shard at a time. Loader accounting reports model/initializer/lazy counts, deterministic bytes validated eagerly, and zero retained source tensors. ## Enforced image-only boundary The checkpoint's source video token is validated, then removed from normalized runtime metadata. Explicit configs requesting video capability fail closed and the embedding graph exposes no `video_features` input. It sanitizes any source video token before vocabulary lookup, computes a runtime `has_video`, and reshapes one zero element to `[1 + has_video]`. Valid requests reduce/add zero; video requests demand two elements and fail ONNX Reshape's element-count invariant before an EP executes. ORT CUDA uses the shared `ReshapeHelper` for this check (`onnxruntime@b1f76d58`, `cuda/tensor/reshape.h`), avoiding CUDA Gather's out-of-range zero-fill behavior. Optimized CPU is exercised locally and the same test executes CUDA when available; processor-native mixed input also fails through the guard. ## Runtime workflow boundary onnxruntime-genai and the current ONNX GenAI workflow schema cannot represent Qwen4Exp's `ple_input_ids`, four-axis positions, and heterogeneous state membership. Both exporters reject before writing artifacts. ONNX GenAI detection checks package config, explicit config, and the structural decoder ABI, so a non-Qwen override cannot bypass preflight or modify an existing output directory. Direct ONNX Runtime users consume `mobius.state_manifest` instead. ## Source pins - `Qwen/Qwen3.8-Flash-Next@f5d08274bafd880402bd16f5e3e6c514136ec06c` - `huggingface/transformers@598d8ba8baaec7fec5a22da0e2844c7bf4ea20e1` ## Final-base rebase Lower-layer PR #656 was squash-merged to `main` as `8f902ac2`. This branch replays only #657's multimodal work onto that commit; no pre-squash lower-layer or GGUF implementation files remain in the PR diff. ## Validation - final Qwen4 loader/multimodal/synthetic suite: 47 passed, 6 skipped - full runtime exporter suites: 263 passed - pinned HuggingFace image parity plus real image/video boundary tests: 3 passed - post-rebase GGUF report/shard/runtime regression suite: 831 passed - Qwen4 component inspection and review-thread regressions: passed - pinned lintrunner format + lint: clean - independent GPT-5.6 Sol CUDA-guard rereview: no findings ## Waivers - **L4/L5 real-weight golden:** the 512-expert checkpoint requires a large GPU; the immutable YAML case records the CI skip. - **Released runtime workflows:** ORT GenAI and ONNX GenAI metadata export fail closed as described above. - **Full checkpoint CUDA generation:** checkpoint scale makes this impractical in this session; CUDA's shared Reshape validation is source-pinned and the guard regression runs on CUDA whenever that EP is available. This PR targets `main`, is intentionally non-draft, and must not be merged by this session. --------- Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
justinchuby
added a commit
that referenced
this pull request
Aug 27, 2026
## Summary - resolve local sharded-GGUF symlinks to their verified regular-file targets before artifact identity hashing, while retaining logical shard filenames in the digest - make missing `split.count`, `split.no`, and `split.tensors.count` diagnostics deterministic - add regressions covering the `build_from_gguf` symlink path, artifact hashing, missing targets, and reversed shard ordering This follows up on the still-applicable Copilot findings from #652 after its sharded import functionality was superseded on `main` by #656. ## Validation - `python -m pytest src/mobius/integrations/gguf/_shard_set_test.py -q --tb=short` (51 passed) - `python -m pytest tests/build_graph_test.py tests/cli_test.py src/ -q -k "not phi4mm and not apply_weights_unknown" --tb=short -n auto` (8627 passed, 64 skipped) - `lintrunner f --output oneline --all-files` - independent code review: no findings --------- Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
justinchuby
added a commit
that referenced
this pull request
Aug 27, 2026
Derive Granite routed expert width from feed_forward_length exactly, reject contradictory serialized expert widths without lossy numeric conversion, and reconcile the authoritative route census with closed #652 plus merged #656 and #675. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 71862fcb-bc9a-4734-b047-b5376afabc3e Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack
justinchuby-finish-qwen4exp-core).eacf244864284cc96bd70f1187b145446783aab4) so the three-file artifact is evaluated as one complete model.Summary
general.architecture=qwen4expasqwen4_exp_textwith the exact Qwen3.8 Flash-Next config, DeltaNet/QSA schedule, hyper-connections, PLE, routed/shared MoE, and output HC mixer metadataunsloth/Qwen3.8-Flash-Next-GGUF@d3bc75ee6ccef3efc1e228ec00a6cc2cdb1e2249, all three shard sizes/LFS SHA-256 identities, and the metadata-only shard-0 closure0 + 595 + 629 = 1224index_qk_proj25a1e6a2073caf19d3a3835dd23702a19fa09cc651506e11a13de7b48076359dFail-closed payload policy
The only published
UD-IQ1_Sroute is intentionally not emitted as an executable ONNX payload:per_layer_token_embd.weightis an enormous IQ4_NL embedding with no compatible native gather ABIMatMulNBitsThe exact header/config/mapping support is therefore a truthful preflight foundation for future ABI work, not a quantized-runtime claim.
Validation
1187 passed, 2 skipped4865 passed, 54 skipped8409 passed, 64 skipped, 12 deselected, 1 subtest passedlintrunner f --output oneline --all-files && lintrunner -a: cleanWaivers