FAEDKV: Infinite-Window Fourier Transform for Unbiased KV Cache Compression
This addresses the bottleneck of biased representations in KV cache compression for LLMs, offering a significant gain in long-context performance without retraining.
The paper tackles the problem of memory and computational demands in Large Language Models for long-context tasks by introducing FAEDKV, a training-free KV cache compression framework that ensures unbiased information retention, achieving up to 22% improvement over existing methods on the LongBench benchmark.
The efficacy of Large Language Models (LLMs) in long-context tasks is often hampered by the substantial memory footprint and computational demands of the Key-Value (KV) cache. Current compression strategies, including token eviction and learned projections, frequently lead to biased representations -- either by overemphasizing recent/high-attention tokens or by repeatedly degrading information from earlier context -- and may require costly model retraining. We present FAEDKV (Frequency-Adaptive Infinite-Window for KV cache), a novel, training-free KV cache compression framework that ensures unbiased information retention. FAEDKV operates by transforming the KV cache into the frequency domain using a proposed Infinite-Window Fourier Transform (IWDFT). This approach allows for the equalized contribution of all tokens to the compressed representation, effectively preserving both early and recent contextual information. A preliminary frequency ablation study identifies critical spectral components for layer-wise, targeted compression. Experiments on LongBench benchmark demonstrate FAEDKV's superiority over existing methods by up to 22\%. In addition, our method shows superior, position-agnostic retrieval accuracy on the Needle-In-A-Haystack task compared to compression based approaches.