Smarter Cache Management for tiflash-compute

[JaySon-Huang]

Enhancement

Currently, TiFlash Compute caches data locally and only evicts entries when the cache reaches its capacity—using a LRU policy to remove the least frequently accessed items. However, TiFlash Compute lacks awareness of whether cached data has become obsolete—for example, due to compaction—meaning it cannot proactively identify and discard data that will never be accessed again.

As a result, a significant amount of useless or stale data may accumulate in the cache. When new data needs to be cached, the system must first evict older entries to make space, which can slow down the downloading and caching of new, relevant data.

Tasks

• A cache eviction API with minimal impact on query requests for tiflash-compute #10619
• Fine grained rate limiting when downloading cache from S3 to local disk #10633