KV-Efficient VLA: A Method of Speed up Vision Language Model with RNN-Gated Chunked KV Cache
This work solves inference inefficiencies critical to real-time deployment of robotic perception and control models, though it is incremental as it builds on existing methods with a novel compression framework.
The paper tackles the scalability issues of Vision-Language-Action models by addressing the quadratic attention cost and unbounded KV memory growth during long-horizon inference, resulting in up to 1.21x inference speedup and 36% KV memory reduction with minimal impact on task success.
Vision-Language-Action (VLA) models promise unified robotic perception and control, yet their scalability is constrained by the quadratic cost of attention and the unbounded growth of key-value (KV) memory during long-horizon inference. While recent methods improve generalization through scaling backbone architectures, they often neglect the inference inefficiencies critical to real-time deployment. In this work, we present KV-Efficient VLA, a model-agnostic memory compression framework that addresses these limitations by introducing a lightweight, training-friendly mechanism to selectively retain high-utility context. Our method partitions the KV cache into fixed size chunks and employs a recurrent gating module to summarize and filter historical context according to learned utility scores. This design preserves recent fine-grained detail while aggressively pruning stale, low-relevance memory, all while maintaining causality. Theoretically, KV-Efficient VLA yields up to 1.21x inference speedup and 36% KV memory reduction, with minimal impact on task success. Our method integrates seamlessly into existing autoregressive and hybrid VLA stacks, enabling scalable inference without modifying training pipelines or downstream control logic.