aphrodite.third_party.deep_gemm._C is built only for cpython-312, breaking DeepGEMM on every other supported Python version (silent fallback, no error surfaced)

[Nakanokensetsu]

Title

aphrodite.third_party.deep_gemm._C is built only for cpython-312, breaking DeepGEMM on every other supported Python version (silent fallback, no error surfaced)

Environment

• aphrodite-engine 0.22.0, installed from PyPI via pip install aphrodite-engine==0.22.0
• Python 3.11.15 (micromamba env)
• torch 2.13.0+cu130, CUDA 13.0
• Linux (WSL2), x86_64

Describe the bug

The aphrodite_engine-0.22.0 wheel is tagged cp38-abi3-linux_x86_64 (stable ABI, Requires-Python: >=3.10,<3.15), and every other compiled extension in the package correctly ships as a .abi3.so (Python-version-independent):

aphrodite/_C_fork.abi3.so
aphrodite/_C_stable_libtorch.abi3.so
aphrodite/cumem_allocator.abi3.so
aphrodite/_flashmla_C.abi3.so
aphrodite/_moe_C_stable_libtorch.abi3.so
...

However, the vendored aphrodite/third_party/deep_gemm/_C extension is not built against the stable ABI — it ships as a single version-specific build:

aphrodite/third_party/deep_gemm/_C.cpython-312-x86_64-linux-gnu.so

On any interpreter other than CPython 3.12 (e.g. 3.10, 3.11, 3.13, 3.14 — all of which this wheel claims to support), import aphrodite.third_party.deep_gemm fails at from . import _C because no matching extension file exists for that ABI tag.

Confusing error message

Because the failing from . import _C happens inside deep_gemm/__init__.py's own top-level code, CPython's import machinery reports this as:

ImportError: cannot import name '_C' from partially initialized module 'aphrodite.third_party.deep_gemm' (most likely due to a circular import)

This is misleading — there is no real circular import. It's simply that no extension file matches the running interpreter's ABI, so _C doesn't exist as an importable submodule at all. This cost significant debugging time before the actual cause (ABI mismatch) was found by comparing .so suffixes against importlib.machinery.EXTENSION_SUFFIXES.

Impact

aphrodite.utils.import_utils.has_deep_gemm() catches the exception and returns False. This degrades gracefully for dense/non-MoE models (no crash, no visible warning at INFO level beyond a one-line warning_once/debug_once), but it silently disables the DeepGEMM-accelerated kernel path for:

• FP8-quantized MoE grouped-GEMM (_valid_deep_gemm() in fused_moe/experts/deep_gemm_moe.py)
• the DeepSeek MLA sparse-attention indexer (v1/attention/backends/mla/indexer.py)
• expert-parallel micro-batch SM budgeting (v1/worker/gpu_ubatch_wrapper.py)
• "mega" MoE dispatch kernels

So any user running an FP8 MoE model (e.g. DeepSeek-V3/V4-style) on Python 3.10/3.11/3.13/3.14 with this wheel silently loses access to DeepGEMM's optimized kernels and falls back to a different backend — with only a debug_once/info_once log line, easy to miss in production.

Reproduction

python3.11 -m venv /tmp/venv311
source /tmp/venv311/bin/activate
pip install aphrodite-engine==0.22.0
python -c "import aphrodite.third_party.deep_gemm"
# ImportError: cannot import name '_C' from partially initialized module ...

Suggested fix

Build third_party/deep_gemm/_C against the stable/limited ABI like the rest of the package's extensions (.abi3.so), or if that isn't feasible for this vendored dependency, build and ship one _C.cpython-3XX-*.so per supported Python version (matching Requires-Python: >=3.10,<3.15) in the release pipeline, the same way manylinux wheels commonly ship multiple version tags.