PruneVid: Visual Token Pruning for Efficient Video Large Language Models
This addresses efficiency challenges for researchers and practitioners using multi-modal video understanding models, though it is incremental as it builds on existing pruning methods.
The paper tackles computational inefficiency in video large language models by introducing PruneVid, a training-free visual token pruning method that reduces redundancy by merging spatial-temporal tokens and selectively pruning features, achieving over 80% token pruning while maintaining competitive performance on benchmarks.
In this paper, we introduce PruneVid, a visual token pruning method designed to enhance the efficiency of multi-modal video understanding. Large Language Models (LLMs) have shown promising performance in video tasks due to their extended capabilities in comprehending visual modalities. However, the substantial redundancy in video data presents significant computational challenges for LLMs. To address this issue, we introduce a training-free method that 1) minimizes video redundancy by merging spatial-temporal tokens, and 2) leverages LLMs' reasoning capabilities to selectively prune visual features relevant to question tokens, enhancing model efficiency. We validate our method across multiple video benchmarks, which demonstrate that PruneVid can prune over 80% of tokens while maintaining competitive performance combined with different model networks. This highlights its superior effectiveness and efficiency compared to existing pruning methods. Code: https://github.com/Visual-AI/PruneVid.