Release v0.4.0 · llm-d/llm-d-kv-cache

Highlights

Unified chat-template rendering interface across the different tokenizers (fixes a BOS token duplication bug), performance improvements and code improvements on templating path by @hyeongyun0916
Logger fix by @hyeongyun0916
Tiered prefix-cache scoring by @Jay-Pd
UDS-tokenizer service by @delavet and @osswangxining
Tokenizer from local disk enablement by @pierDipi
Valkey kvblock backende by @rishi-jat
General enhancements by: @zhengkezhou1 @Frapschen @my-git9 @samzong @samber @kyanokashi

Fix examples precalculated hashes by @vMaroon in #141
feat: make kvcache.Indexer as gRPC Service by @zhengkezhou1 in #109
Fix: division by zero in metrics logging by @yankay in #145
feat: Add Valkey and RDMA support for KV-cache indexing by @rishi-jat in #139
feat: Add UDS-based external tokenizer service by @delavet in #137
Fix LookupHits metrics not work by @Frapschen in #146
[feat] Add tolerations for chart by @my-git9 in #149
[misc] valkey doc repositioning by @vMaroon in #154
Update README.md to enhance clarity of flowchart labels and descriptions by @samzong in #157
perf(prefixstore): sync.Map are much faster in read-intensive applications by @samber in #156
Add support for local tokenizer files by @pierDipi in #142
Add @delavet as /services/uds_tokenizer owner by @vMaroon in #164
Add @osswangxining as /services/uds_tokenizer owner by @vMaroon in #168
Implementation for Tiering in KV-Cache-Manager by @Jay-Pd in #150
fix: rename LookupHits metric to MaxHitsPerPod to better reflect what's tracked by @kyanokashi in #160
fix online example chart format by @delavet in #171
Minor fix for KV Device Tier by @Jay-Pd in #172
[Fix] Ensure Correct Logger Usage by Replacing klog.FromContext with log.FromContext by @hyeongyun0916 in #169
refactor(tokenizer): Unify interface for RenderChatTemplate and eliminate object creation overhead by @hyeongyun0916 in #163
Minor logger fix by @vMaroon in #173
General refactoring for v0.4.0 by @vMaroon in #174

Full Changelog: v0.3.2...v0.4.0