Scholar Feed

KV-cache compression

EPIC

EPIC: Efficient Position-Independent Caching for Serving Large Language Models

Superseded baseline#55 of 234 most-superseded · first seen Oct 20, 2024

Superseded — cited as a baseline and beaten by newer methods

2 papers critique it · 1 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites EPIC as a baseline.

Nonetheless, existing systems (, vLLM, CacheBlend, CacheCraft, Epic)~kwon2023efficient,yao2025cacheblend,agarwal2025cache,hu2024epic operate at coarse granularity and thus fundamentally cannot support selective sharing: they reuse KV cache at the level of fixed chunks (e.g., 512 tokens~agarwal2025cache) or entire prompt, so the presence of a single sensitive token (e.g., PII) invalidates the whole unit and discards most otherwise reusable content
— CachePrune: Privacy-Aware and Fine-Grained KV Cache Sharing for Efficient LLM Inference
“rely on exact context matching, which is unsuitable for real user scenarios”
— SemShareKV: Efficient KVCache Sharing for Semantically Similar Prompts via Token-Level LSH Matching

Beaten on benchmarks

Head-to-head results where a newer method reports beating EPIC. Values are copied from the source paper's tables — verify against the cited paper.

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.