fix(vllm): preserve duplicate prefix cache blocks

[shipiyouniao]

Summary

Preserve every physical vLLM prefix-cache block when concurrent requests materialize the same (block_hash, group_id).

Closes #401.

Root cause

ElasticBlockPool._cached_blocks currently maps each cache key to one KVCacheBlock. cache_full_blocks() treats an existing key as an idempotent call and skips the new block.

Concurrent requests can produce the same full prefix before either physical block becomes reusable. In that case the second block is distinct, not an idempotent repeat. Skipping it leaves KVCacheBlock.block_hash unset and omits its reverse-index entry. vLLM's later cache accounting can then fail at assert block.block_hash is not None, terminating the EngineCore.

This complements #363: that PR fixed hash set/reset lifecycle, while this PR preserves multiple physical blocks that legitimately share one hash.

Changes

• change the prefix index from cache key -> block to cache key -> {block_id: block}
• keep lookup compatibility by returning one matching block
• remove and reset cached entries by exact block_id, so evicting one duplicate does not delete its siblings
• handle a reused block moving from an old cache key to a new one
• add a regression test for two blocks with the same prefix, followed by separate release and eviction

Validation

• python -m pytest tests/test_prefix_cache.py -q -> 30 passed
• complete pre-commit run --all-files -> all hooks passed
• manual mypy matrix for Python 3.9, 3.10, 3.11, 3.12, and 3.13 -> all passed
• high-concurrency two-instance vLLM integration replay with prefix caching -> 1440/1440 requests completed, with no block-hash assertion or EngineCore failure

The branch is based on current upstream main (927ef66c) and contains one focused commit.

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[Copilot]

Pull request overview

This PR fixes a correctness issue in the vLLM prefix-cache integration where concurrent requests can materialize the same prefix into multiple distinct physical blocks, but the cache index previously collapsed them into a single entry. The change preserves all physical blocks that share a (block_hash, group_id) so every block’s block_hash metadata and reverse-index bookkeeping remains consistent, preventing downstream vLLM assertions from firing.

Changes:

• Change the prefix cache index from key -> block to key -> {block_id: block} while keeping get_cached_block() compatibility by returning one matching block.
• Update eviction/removal paths to remove cached entries by exact block_id so evicting one duplicate does not delete its siblings.
• Add a regression test covering two blocks sharing a prefix hash, followed by independent release and eviction.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
kvcached/integration/vllm/patches.py	Refactors prefix-cache bookkeeping to store multiple physical blocks per cache key and evict/reset by block_id.
tests/test_prefix_cache.py	Adds a regression test ensuring duplicate in-flight prefixes preserve metadata and only the evicted block is reset.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.