Scholar Feed

KV-cache compression

ShadowKV

ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference

Superseded baseline#14 of 234 most-superseded · first seen Oct 28, 2024

Superseded — cited as a baseline and beaten by newer methods

4 papers critique it · 6 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites ShadowKV as a baseline.

However, limited by the accuracy constraints of intra-layer SVD, ShadowKV is forced to offload Value states to the CPU, leaving inference speed bounded by PCIe bandwidth.
— xKV: Cross-Layer SVD for KV-Cache Compression
“We systematically evaluate KV offloading methods on context-intensive tasks and observe significant accuracy drops”
— KV Cache Offloading for Context-Intensive Tasks
“ShadowKV does not support long-generation since the SVD is performed only once during prefill, leaving the low-rank key unupdated during decoding.”
— FreeKV: Boosting KV Cache Retrieval for Efficient LLM Inference
“Unfortunately, their coarse-grained retrieval strategies often overlook fine-grained dependencies and incur high I/O overhead.”
— HeteroCache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inference

Beaten on benchmarks

Head-to-head results where a newer method reports beating ShadowKV. Values are copied from the source paper's tables — verify against the cited paper.

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.