Prove exact Apertus GGUF runtime support - #683
Merged
Merged
Conversation
Performance Comparison
|
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Pull request overview
Promotes a single pinned Apertus v1.1 1.5B Instruct BF16 GGUF (CPU, exact-float import) to runtime-supported status by adding immutable runtime evidence records, expanding config parsing to match the pinned GGUF metadata conventions, and updating the evidence/docs/test harnesses to prove and document the support boundary.
Changes:
- Add Apertus promoted runtime integration coverage (full-logit parity + deterministic ORT GenAI generation assertions) and an independent raw GGUF reader oracle path.
- Extend Apertus GGUF config post-processing to accept pinned llama.cpp-style unqualified
xielu.*metadata and a factorless default RoPE, while tightening/clarifying RoPE scaling subset validation. - Register Apertus runtime evidence as “supported”, and update generated capability artifacts + documentation tables accordingly.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/gguf_small_model_runtime_integration_test.py | Adds a promoted Apertus runtime case plus oracle mapping for Apertus-specific weights/metadata. |
| testdata/evidence/gguf_quantization_capabilities.json | Adds immutable Apertus runtime evidence record and updates selected artifact budget totals. |
| testdata/cases/schema.json | Raises per-case download budget ceiling to accommodate large real-artifact GGUF cases. |
| testdata/cases/causal-lm/apertus-v1.1-1.5b-instruct-bf16.yaml | Adds the pinned Apertus BF16 GGUF runtime-evidence enrollment case. |
| src/mobius/integrations/gguf/_runtime_evidence.py | Introduces structured Apertus runtime evidence record and registers it for lookup. |
| src/mobius/integrations/gguf/_route_census_test.py | Updates census expectations to reflect Apertus no longer being “evidence-only”. |
| src/mobius/integrations/gguf/_quant_capabilities_test.py | Updates artifact count/byte totals to include Apertus in the selected real artifacts set. |
| src/mobius/integrations/gguf/_docs_test.py | Ensures docs enumerate Apertus as runtime-supported with structured evidence. |
| src/mobius/integrations/gguf/_config_mapping.py | Adjusts Apertus metadata handling (generic xielu.*, factorless RoPE, closed scaling subset validation). |
| src/mobius/integrations/gguf/_arch_registry.py | Promotes Apertus runtime to supported and wires runtime evidence ID + scoped reason. |
| src/mobius/integrations/gguf/_apertus_test.py | Adds focused Apertus tests for generic metadata acceptance, factorless RoPE, and rejection paths. |
| docs/api/build_from_gguf.md | Updates generated support tables to reflect Apertus runtime support and evidence listing. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
justinchuby
force-pushed
the
justinchuby-exact-apertus-gguf
branch
2 times, most recently
from
August 28, 2026 20:16
8cd4b29 to
7e9b199
Compare
Promote the pinned Apertus v1.1 1.5B BF16 route with independent raw-GGUF full-logit and stateful decode evidence. Preserve exact tokenizer provenance and package fingerprints while failing closed on conflicting xIELU and RoPE metadata. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 75ccb9e3-3ba3-4bee-b5ff-c395a88af1ca Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Regenerate Apertus against the final runtime-package schema and preserve the serialized graph identity independently from export metadata. Allow only exact final-package evidence to supersede an identifier-level tokenizer census blocker, while unmatched artifacts remain advisory and omit unverified tokenizer assets. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 75ccb9e3-3ba3-4bee-b5ff-c395a88af1ca Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Promote a pre-existing deferred tokenizer component to exported when immutable final-package runtime evidence materializes its pinned source. Regenerate the Apertus package identity and assert that the resulting report is complete, validated, and end-to-end runnable after the #697 tokenizer-policy rebase. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 75ccb9e3-3ba3-4bee-b5ff-c395a88af1ca Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
justinchuby
force-pushed
the
justinchuby-exact-apertus-gguf
branch
from
August 28, 2026 21:09
7e9b199 to
4aed1b6
Compare
Match immutable final-package evidence without requiring callers to repeat its pinned tokenizer source. Preserve matching explicit overrides, reject conflicting tokenizer identities before serialization, and keep unmatched artifacts deferred and tokenizer-omitted. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 75ccb9e3-3ba3-4bee-b5ff-c395a88af1ca Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
justinchuby
added a commit
that referenced
this pull request
Aug 28, 2026
Adds NVIDIA's [RE-USE](https://huggingface.co/nvidia/RE-USE) speech-enhancement generator (SEMamba) on current Mobius APIs. This is a semantic replay of the original PR head `809d674deb6003892e1b439fd0e00ae64abae322` onto `main` at `027fd42bb89e970defde0098792df104cd83b541`, pinned to NVIDIA source/config/weights revision `761905064ea1ea882e015e20a64e2e9d28458890`. RE-USE is not a Transformers architecture: its config has no `model_type`, and the checkpoint targets `mamba_ssm`. Mobius therefore detects the published SEMamba config structurally and supports both the normal API/CLI model source path and local `--config` directories. ## Implementation - `SequenceSelectiveScan`: stateless Mamba-1 recurrence over a full sequence as one ONNX `Scan`, initialized to zero for every graph invocation with dynamic batch shape. Decay/state accumulation and `softplus` stay in FP32; outputs return to the activation dtype. - `SequenceMambaBlock`: full-sequence counterpart to decode-time `MambaBlock`, retaining checkpoint-compatible parameter names. - RE-USE model: dense convolutional encoder, 30 bidirectional time/frequency SEMamba blocks, separate magnitude and phase decoders, and ONNX `atan2` composition. - `SpeechEnhancementTask`: `noisy_mag`/`noisy_pha` inputs and `denoised_mag`/`denoised_pha`/`denoised_com` outputs, with dynamic batch and time dimensions. - onnx-genai workflow metadata: NVIDIA preprocessing contract (`center_reflect_unnormalized`, Hann window, hop 40, window/FFT 320, 8 kHz, `log1p`) plus the required audio-preprocess adapter ABI. - Public build/CLI integration: `mobius.build()`, `mobius build --model ...`, and `mobius build --config ...` all recognize the bespoke checkpoint. Remote exports default to the immutable NVIDIA revision; local exports report local provenance. The default export now preserves NVIDIA native-rate behavior: frequency is dynamic, the adapter keeps the decoded sample rate, and FFT/hop/window use the pinned source floor-then-even scaling rule. `--input-sample-rate HZ` explicitly bakes a static native-rate graph without resampling; `--bwe-sample-rate HZ` explicitly applies NVIDIA BWE resampling and bakes matching static geometry. Unsupported adapters must reject the declared `scaled_spectrogram` operation rather than silently analyzing native audio with 8 kHz geometry. ## Source and weight validation The pinned NVIDIA generator, codec, TF-Mamba block, STFT, inference code, and published config were checked directly. The graph preserves: - all 30 TF-Mamba blocks and their bidirectional time/frequency recurrence; - fresh zero recurrence state per invocation, with no hidden cache carried between calls; - reference convolution geometry, normalization, crop behavior, and complex-output construction; - external STFT/ISTFT boundaries and the published magnitude compression contract. All **1,416 checkpoint tensors** map to all **1,416 Mobius parameters** after preprocessing: no missing, extra, or shape-mismatched weights. The built graph has 1,429 initializers and 5,207 nodes. A full 9.61M-parameter weighted CPU run serialized to 42,462,537 bytes, had no unfilled initializers, accepted `(1, 161, 1)`, and produced finite nonzero outputs with shapes `(1, 161, 1)`, `(1, 161, 1)`, and `(1, 161, 1, 2)`. The original PR's independent pure-PyTorch transcription comparison with real weights remains valid evidence: magnitude and complex outputs matched to **4.2e-06**; phase exceeded `1e-04` only at 10 of 6,601 near-zero-radius points where `atan2` is ill-conditioned, with complex-plane error at those points at most `1.7e-05`. ## Dtype and execution-provider behavior - FP32 and FP16 execute on CPU across dynamic time lengths. - BF16 exports successfully. Stock CPU ORT has no BF16 `Conv(22)` implementation, which is a downstream kernel limitation and does not block export. - Default, CPU, CUDA, and DML EP-aware graph construction succeeds. CUDA/DML runtime execution was unavailable on the validation host, so no runtime claim is made for those providers. - Export is not gated on downstream onnxruntime-genai or plugin-EP support. ## Preserved MLX evidence The time-major `Scan` layout is retained because `onnxruntime-ep-mlx` 0.29.4 did not claim non-axis-zero Scan. The original measurements were on an Apple M1 Max using 2 seconds of 8 kHz audio in steady state: approximately **7x** over CPU. The first MLX invocation was roughly **4x slower than steady state** because of graph translation/kernel compilation. These historical measurements are performance evidence, not an export gate, and should be cited with their hardware, runtime version, input geometry, and warm-up conditions. ## Validation - Focused RE-USE model, SSM/Mamba component, metadata, API, and CLI tests pass. - Broad serial suite: **9,741 passed, 64 skipped, 12 deselected, 47 warnings, 1 subtest**. - Initialized lintrunner: clean. - Focused mypy for changed model/task and SSM/Mamba components: clean with third-party imports skipped. - Generated 368 model pages plus the index: no resulting documentation diff. - Independent medium code review completed; findings around `--config` detection, task-object acceptance, and local revision provenance were fixed and retested. The standard Transformers config-extraction and Transformers integration levels do not apply because NVIDIA RE-USE is a bespoke non-Transformers checkpoint; direct config parsing, pinned-source inspection, exact real-weight loading, full weighted ORT execution, and reference-transcription parity cover those semantics instead. Autoregressive generation testing does not apply to a single-invocation speech-enhancement graph. ## Final exact-base freeze Semantically rebased onto current `main` at `2c981f2b631a907f43c57d3235647c22cc888eae` (#697). Its GGUF tokenizer-evidence and runtime-identity changes are disjoint and preserved. Native-rate executable workflow evidence uses the published 8 kHz baseline `(n_fft=320, hop=40, window=320)` and verifies exact scaled geometry at 8/16/48 kHz: `(320,40,320)` / `(640,80,640)` / `(1920,240,1920)`, producing 161/321/961 graph input bins. Odd and even waveform lengths retain their native rate and exact input length after inverse STFT alignment. Explicit BWE from 8 kHz to 16 kHz returns 16 kHz audio at the resampled length with `(640,80,640)` geometry and 321 bins. The native-rate and BWE selections are fail-closed and mutually exclusive across every public surface: `ReUseConfig.validate()`, direct `build_reuse()`, public `mobius.build()`, top-level `write_onnx_genai_config()`, direct workflow-metadata construction, metadata writing, and CLI parsing. Top-level auto-export recognizes `ReUseConfig` from either the explicit argument or `pkg.config` and validates before package/graph inspection, structural dispatch, advisory early returns, incomplete-STFT fallback, destination creation, or writes. Equal and unequal dual-rate pairs reject even with incomplete/empty packages and missing reference geometry; strings, floats, booleans, zero, and negative rates reject; valid no-rate and single-rate incomplete geometries retain the intentional spectrum-input fallback. Semantically rebased onto current `main` at `40c8469cf58275be554f793c9586d31dba51b369` (#683). Its Apertus GGUF runtime-identity, evidence, config-mapping, schema, and documentation changes are disjoint and preserved. Final exact-head gates: affected suite **940 passed, 275 skipped, 1,770 deselected**; broad serial suite **9,833 passed, 64 skipped, 12 deselected, 47 warnings, 1 subtest**; initialized lintrunner, 368 generated model pages/index, focused RE-USE mypy, and diff checks are clean. An independent medium review traced every public auto-export/workflow entry and early return, verified absent destination side effects, audited the full RE-USE diff and Apertus reconciliation, and reported no findings. Head: `54a8f589d00eddca6c1ed93b629639801cad5fc4` Base: `40c8469cf58275be554f793c9586d31dba51b369` --------- Signed-off-by: Copilot <justinchuby@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 43bb31ac-c136-44bc-8cdf-70cbcedd4a42 Copilot-Session: cb70334e-9510-469a-bd3c-f1301eb0ae47
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
xielu.*metadata and factorless default RoPE, while rejecting conflicting xIELU keys, incomplete/malformed factors, unsupported scaling metadata, ignoredoriginal_context_length, and lossy factor storage.export_report.json, then validate the complete final runtime package including that report.gguf.GGUFReader+dequantizeoracle with Mobius loading, mapping, tensor-processing, and normalization helpers patched to raise.Immutable evidence
MrMeOrYou/Apertus-v1.1-1.5B-Instruct-GGUF@88c75ad49566d3c2157d03709bf772262c3241edApertus-v1.1-1.5B-Instruct-BF16.gguf3,028,052,608/f9ec154d0ec29dad1f6465b458b7f27bd25ad7b9a3899233ae98ca6d358501c2swiss-ai/Apertus-v1.1-1.5B-Instruct@9e9d01154446a645d30f04174cf1515a38058be7BF16: 98,F32: 65)4bb91bade19d41559cb524e28692453851fe67274cc58109787ee968df3e0fe597582549e9c5b4114f3bcfa81c92f16aba9d14dd0ea0d2b20acaefd9060e6486The real-artifact test invokes runtime packaging without tokenizer source flags and proves automatic evidence-pinned materialization, full-logit prefill, fresh and cached decode parity, full-sequence replay, rollback, batch reorder, 20 exact greedy decode steps, deterministic ORT GenAI generation, tokenizer vocabulary/special-token/chat-template identity, graph/config/package fingerprints, package/report roundtrip, complete validated export reporting, and fail-closed artifact/source/tokenizer/runtime mismatches. Observed prefill max absolute difference was
0.00016784668; cached-vs-fresh differences remained below4e-5.Truthful support boundary
source_fidelity=truebecause BF16 values are exactly representable after import to float32.storage_quantized=falsebecause the emitted ONNX package stores float tensors. Runtime support is evidence-gated to this exact artifact, official tokenizer revision, CPU import route, ONNX Runtime 1.29.0, and ORT GenAI 0.15.2. Other Apertus artifacts retain the generic best-effort advisory behavior and cannot inherit this evidence. This does not claim a generic independent Tekken tokenizer promotion, preserved-quantized runtime, CUDA, or other runtime versions. No Apertus model-ID/revision production gate was added, and the #675 source/import-route identity remains intact.Validation
python -m pytest tests/build_graph_test.py tests/cli_test.py src/ -q -k 'not phi4mm and not apply_weights_unknown' --tb=short— 9,675 passed, 64 skipped, 12 deselected, 1 subtest passedlintrunner f --output oneline --all-files && lintrunner -a— cleanpython scripts/generate_gguf_support_docs.py --check— currentWaivers
Semantically rebased and validated at exact base
2c981f2b631a907f43c57d3235647c22cc888eae; exact head256c2b15f17ec327634ef5cae1d1e902a07f5309. Do not merge as part of this task.