fix(sglang): use storage dtype for FP8 KV pools

[shipiyouniao]

Summary

Closes #387.

Use SGLang's physical KV-cache store_dtype consistently when kvcached constructs MHA and MLA backing pools.

Root cause

SGLang separates the model-facing logical dtype from the physical store_dtype. FP8 KV storage can therefore use one-byte physical elements while the logical dtype remains wider. The kvcached patches used the logical dtype for tensor allocation, page cell geometry, and physical byte accounting, allowing the allocator contract and the actual storage layout to disagree.

Changes

• allocate MHA and MLA backing KV tensors with self.store_dtype
• derive MHA and MLA allocator cell sizes from store_dtype.itemsize
• report physical MHA and MLA bytes using store_dtype.itemsize
• add focused regression tests where logical and storage dtypes intentionally differ

The logical self.dtype remains unchanged for model-facing behavior.

Validation

• python -m pytest tests/test_sglang_storage_dtype.py -q -> 2 passed
• complete pre-commit run --all-files under Python 3.11 -> all hooks passed
• manual mypy matrix under Python 3.9, 3.10, 3.11, 3.12, and 3.13 -> all passed
• CI containers used standard runc with GPU devices explicitly hidden

This is independent from vLLM hybrid mapping and scheduler policy.

[Copilot]

Pull request overview

This PR fixes a dtype/layout mismatch in the SGLang integration by consistently using SGLang’s physical KV-cache store_dtype (instead of the logical dtype) when kvcached allocates and accounts for MHA/MLA KV backing buffers—important for FP8 storage where physical element width differs from the logical dtype.

Changes:

• Update SGLang MHA/MLA pool patches to size allocator cells and physical byte accounting using store_dtype.itemsize.
• Allocate MHA/MLA backing KV buffers using store_dtype.
• Add regression tests covering the case where logical and storage dtypes intentionally differ.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
kvcached/integration/sglang/patches.py	Switches MHA/MLA physical allocation + sizing/accounting to store_dtype semantics.
tests/test_sglang_storage_dtype.py	Adds targeted regression tests for logical-vs-storage dtype divergence.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.