Skip to content

Fix vLLM Gemma4 support by adding v2 module fallback and patch target fallback - #386

Open
mahendrarathore1742 wants to merge 4 commits into
ovg-project:mainfrom
mahendrarathore1742:fix/gemma4-vllm-v2-compat
Open

Fix vLLM Gemma4 support by adding v2 module fallback and patch target fallback#386
mahendrarathore1742 wants to merge 4 commits into
ovg-project:mainfrom
mahendrarathore1742:fix/gemma4-vllm-v2-compat

Conversation

@mahendrarathore1742

Copy link
Copy Markdown
Contributor
  • Added support for vLLM v2 module layout fallback in vLLM integration patches.
  • Updated PatchManager to accept multiple target modules and try alternate import candidates before failing.
  • Added fallback-aware imports for vllm.v1.* and vllm.v2.* in patches.py and nixl_compat.py.
  • Added regression coverage in test_vllm_v2_support.py to verify vLLM import fallback and patch manager module selection.
  • This resolves Gemma4 model loading issues caused by newer vLLM engine path changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant