Repository navigation
Conversation
…cee-ai#708) A shared role-key space so parameters from different architecture families can be addressed uniformly before any merge method runs: model.layers.0.self_attn.q_proj.weight -> layer_0.attn_q.weight model.layers.11.mlp.up_proj.weight -> layer_11.ffn_up.weight Dependency-free name mapping only: values pass through by reference, no tensor ops, no new requirements. Ships the Llama-family detector (Llama, Qwen, Mistral, Gemma, TinyLlama layouts) with detection, canonicalization, and an exact decanonicalize inverse (round-trip tested). Remaining families (gpt2, bert, neox, opt, t5, phi) and the shape bridge follow in the same pattern per arcee-ai#708. The detector set is exercised end-to-end by the cross-family merge pipeline behind the published Optitransfer converge collectives (9 models across 4 architecture families in one set of weights). Signed-off-by: Ryan Gillespie <mgillr@users.noreply.github.com>
BertFamilyDetector covers BERT and RoBERTa encoder layouts (post-norm LayerNorms mapped onto the post_attn_norm / post_ffn_norm canonical slots); OPTDetector covers the OPT decoder layout. Same pattern as the Llama-family detector in arcee-ai#709: pure name mapping, values by reference, no new dependencies. Cross-detector exclusivity tested (a Llama-layout dict matches neither). Signed-off-by: Ryan Gillespie <mgillr@users.noreply.github.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit ae93097. Configure here.
…t import branches Signed-off-by: Ryan Gillespie <mgillr@users.noreply.github.com>
…r in CI) Signed-off-by: Ryan Gillespie <mgillr@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Stacks on #709 (contains its commits; review the second commit for the delta).
Adds two more pure-name detectors in the same pattern:
BertFamilyDetector— BERT and RoBERTa encoder layouts; post-norm LayerNorms map onto thepost_attn_norm/post_ffn_normcanonical slots (schema extended accordingly, plusembed_positionsand the norm/head bias globals).OPTDetector— the OPT decoder layout (model.decoder.layers.N.*), distinguished from the Llama layout that architecture: canonical parameter-name mapping (first increment of #708) #709's detector already rejects.Tests: per-family mapping with value identity, all outputs pass
is_canonical, and cross-detector exclusivity (a Llama-layout dict matches neither new detector). gpt2/neox/phi remain for the third increment — they need fused-QKV splits, which are tensor ops and belong with the shape-bridge increment rather than this dependency-free module.Note
Low Risk
Additive, dependency-free name remapping with unit tests only; no changes to existing merge execution paths.
Overview
Extends the cross-architecture canonical name space (#708) with two more pure-name detectors alongside the existing Llama-family mapper:
BertFamilyDetector(BERT/RoBERTa encoder keys, including post-norm LayerNorms mapped topost_attn_norm/post_ffn_norm) andOPTDetector(model.decoder.layers.*, kept distinct from Llama viamodel.decoder.detection).The shared schema gains
post_attn_normandpost_ffn_normroles plus globals forembed_positions,final_norm.bias, andhead_lm.biasso OPT and encoder layouts can round-trip into the samelayer_N.role.weightkeys without tensor ops.Tests cover BERT/OPT detect + canonicalize (value identity,
is_canonical), a regression that OPT per-layerfinal_layer_norm.biasmaps topre_ffn_norm.bias, and cross-detector exclusivity on Llama-layout state dicts.Reviewed by Cursor Bugbot for commit f3b02aa. Bugbot is set up for automated code reviews on this repo. Configure here.