Last But Not Least: Boundary Attention CalibratiON for Multimodal KV Cache Compression
Reduces decoding latency for multimodal large language models by improving KV cache compression, addressing a bottleneck in long visual context processing.
BACON calibrates observation window attention with last-query evidence to improve multimodal KV cache compression, achieving 7.5% average improvement under aggressive budgets and up to 30.9% gain.
Multimodal Large Language Models (MLLMs) achieve strong vision-language reasoning, but long visual contexts enlarge the KV cache and increase decoding latency. Existing compression methods rely on observation window attention for stable token-importance estimation, yet this aggregation can dilute sparse visual evidence and discard answer-critical tokens under aggressive compression. Therefore, we identify last-query attention as a complementary source for recovering such evidence, but its answer-irrelevant signals can mislead retention. We propose BACON, a plug-and-play method that calibrates observation window attention with last-query evidence and suppresses isolated noise via intra-layer coherence and inter-layer persistence. Across diverse benchmarks, models, budgets, and compression methods, BACON improves multimodal KV compression by 7.5% on average under the most aggressive budget, with gains up to 30.9%.