Fix ONNX GenAI special-token authority - #644
Merged
Merged
Conversation
Performance Comparison
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates Mobius’s ONNX GenAI inference-metadata producer to make numeric special-token IDs authoritative package facts under package.tokenizer.special_tokens, while removing duplicated/literal EOS defaults from workflows and aligning the local JSON schema to the corresponding onnx-genai metadata proposal.
Changes:
- Migrate
package.tokenizer.special_tokensfrom text-bearingSpecialTokenFactobjects to numericTokenFacts(including ordered multi-EOS), while keepingSpecialTokenFactimport-compatible. - Remove authored
package.eos_idsliterals and make workflow EOS tensors runtime/request-bound inputs (eos_token_ids/eos_token_lengths), including VLM + decoder workflows. - Narrow CTC package token facts to the padding/blank relationship and bump schema stamping to
v1.2where the new fields are emitted.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/mobius/integrations/onnx_genai/workflow_metadata.py | Removes package.eos_ids literals; makes EOS request/runtime-bound; restricts CTC token roles; bumps schema_version where needed. |
| src/mobius/integrations/onnx_genai/workflow_metadata_test.py | Updates VLM test to assert EOS token IDs live under package.tokenizer.special_tokens and package.eos_ids is absent. |
| src/mobius/integrations/onnx_genai/package_facts.py | Introduces TokenFacts numeric special-token facts; preserves ordered multi-EOS; updates tokenizer facts emission and stamps schema v1.2 when attaching facts. |
| src/mobius/integrations/onnx_genai/package_facts_test.py | Updates tests to numeric token facts, multi-EOS ordering, placeholder roles, CTC pad-only facts, and legacy import compatibility. |
| src/mobius/integrations/onnx_genai/auto_export_test.py | Adjusts dispatch expectations for runtime-role EOS inputs. |
| src/mobius/integrations/onnx_genai/_schema/inference_metadata.schema.json | Syncs schema: replaces SpecialTokenFact with TokenFacts, relaxes tokenizer required fields, clarifies EOS override roles. |
| src/mobius/integrations/onnx_genai/init.py | Re-exports TokenFacts to keep public API aligned with new producer types. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
172
to
+176
| def to_metadata(self) -> dict[str, Any]: | ||
| facts: dict[str, Any] = { | ||
| "algorithm": self.algorithm, | ||
| "vocab_size": self.vocab_size, | ||
| "byte_level": self.byte_level, | ||
| } | ||
| facts: dict[str, Any] = {"byte_level": self.byte_level} | ||
| if self.algorithm is not None: | ||
| facts["algorithm"] = self.algorithm | ||
| if self.vocab_size is not None: |
Comment on lines
583
to
584
| address nothing, so the vocabulary is the width a caller can actually render. | ||
| """ |
justinchuby
force-pushed
the
justinchuby/special-token-authority
branch
from
August 26, 2026 05:41
ed33bb3 to
408013b
Compare
Member
Author
|
The metadata-specific failures are resolved: regenerated decoder/static-cache/VLM fixtures now match schema v1.2, and the integration subtree passes locally (566 tests). The remaining cross-platform test failures reproduce on |
Member
Author
|
Producer branch refreshed and validated:
Merge order remains deliberate: merge #647 first, then the onnx-genai consumer schema/runtime in justinchuby/onnx-genai#2171, then rebase this producer PR on current |
justinchuby
force-pushed
the
justinchuby/special-token-authority
branch
from
August 26, 2026 06:43
408013b to
031ee3b
Compare
justinchuby
added a commit
to justinchuby/onnx-genai
that referenced
this pull request
Aug 26, 2026
…2171) ## Summary - establish `package.tokenizer.special_tokens` as the sole package authority for execution-relevant numeric token IDs - preserve ordered multi-EOS defaults while treating request EOS values as replacement overrides - require complete-generation workflows to execute termination semantics; keep logits-only packages explicitly valid without sampler/termination claims - materialize one effective token set for workflow graphs and fused/native runtime paths - document capability obligations, portable state invariants, policy graphs, ONNX ABI versus workflow authority, and reserved versus extensible identifiers - validate and publish schema-v1.2 canonical plus annotated metadata across the hosted collection ## Contract Token IDs are package facts under `package.tokenizer.special_tokens`. Token spellings, added-token mappings, and chat templates remain authoritative in tokenizer assets. Workflow ports route semantic values but do not duplicate ONNX physical ABI, and termination policy consumes resolved token facts without owning another numeric copy. Request EOS values replace package defaults when supplied. Pre-v1.2 compatibility may derive EOS from legacy tokenizer assets; v1.2 packages fail closed on retired workflow/package EOS literals. ## Reader documentation - [Metadata capability model](https://github.com/justinchuby/onnx-genai/blob/justinchuby/special-token-authority/docs/genai/METADATA_CAPABILITY_MODEL.md) - [Normative inference metadata decisions](https://github.com/justinchuby/onnx-genai/blob/justinchuby/special-token-authority/docs/genai/INFERENCE_METADATA_DECISIONS.md) - [Reader tracking issue #2143](#2143) ## Producer and hosted examples - Mobius producer migration: onnxruntime/mobius#644 - Mobius baseline coverage repair: onnxruntime/mobius#647 - Hosted collection: **28 repositories / 56 canonical+annotated files**, including 12 complete-generation packages migrated to schema v1.2 - Catalogue revision: [`8ba416600109201e10256841e20f0c1d2777af6e`](https://huggingface.co/datasets/justinchuby/onnx-genai-inference-metadata-catalogue/tree/8ba416600109201e10256841e20f0c1d2777af6e) ## Validation - metadata and genai-config test suites - engine authored-workflow and multi-EOS tests - ORT tokenizer/chat-template tests - server multimodal tests and Rust 1.98 Clippy with warnings denied - generated schema synchronization, formatting, and `git diff --check` - collection validator: semantic YAML equality, inline comments, README links, provenance hashes, generation scope, token authority, executable termination, schema, and shapes for all 56 files The non-required **Mobius metadata packages (signal)** job is expected to reject current Mobius `main` fixtures until producer PR #644 lands; v1.2 validation is intentionally not weakened for that transition. --------- Signed-off-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
Author
|
Coordination update: |
Member
Author
|
Rebuild preview update (not pushed yet): I rebuilt this producer change on current
The preview also exposed one real cross-change issue: #649's batch-capacity helper unconditionally assigned schema v1.1 after token facts had already selected v1.2. The prepared fix upgrades only exact legacy Validation on the clean preview:
I am intentionally holding the force-push until #647 merges, so its coverage repair is inherited from |
Emit schema-v1.2 numeric token facts under package.tokenizer.special_tokens, preserve ordered multi-EOS defaults, and make decoder/VLM EOS inputs runtime-bound request overrides instead of package workflow literals. Cover image, video, and audio placeholders, keep CTC limited to its pad/blank fact, synchronize the consumer schema, and retain the prior public SpecialTokenFact import for compatibility. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: justinchuby <justinchuby@users.noreply.github.com>
Regenerate decoder, static-cache, and VLM conformance metadata with schema-v1.2 package tokenizer facts and request-bound EOS roles. Also apply the formatter change required by the current main branch lint gate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: justinchuby <justinchuby@users.noreply.github.com>
Keep schema v1.2 after declaring batch capacities and resolve the decoder fixtures against the latest batching contract. This prevents the batching helper from downgrading token-authority metadata emitted by the producer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: justinchuby <justinchuby@users.noreply.github.com>
justinchuby
force-pushed
the
justinchuby/special-token-authority
branch
from
August 26, 2026 13:33
031ee3b to
fd1689b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
package.tokenizer.special_tokenspackage.eos_idsandpackage.eos_token_idsliterals from decoder and VLM workflowsSpecialTokenFactcompatibilityContract
Token IDs are package facts. Token spellings, added-token mappings, and chat templates remain authoritative in tokenizer assets. Termination policy consumes resolved package defaults or an explicit request replacement but owns no duplicate IDs.
Consumer schema/runtime: justinchuby/onnx-genai#2171 (merged as
e599dc0f067d35aa5326ce5c97d83d2fd6d02f78)Reader guide: justinchuby/onnx-genai#2143
Baseline coverage repair: #647
Validation
src/mobius/integrations/onnx_genaiandtests/model_coverage_test.pylintrunnergit diff --checkThis branch is temporarily stacked on #647 solely to avoid known current-
maincoverage failures. After #647 merges, rebase this PR onto currentmainso only the two producer commits remain, rerun the focused gates, and merge.