Spatial-Aware Reduction Framework: Towards Efficient and Faithful Visual State Space Models
This work solves the problem of efficient token reduction for vision Mamba models, which previously suffered from severe performance degradation due to spatially agnostic methods.
STORM addresses the performance collapse in token reduction for visual state space models by introducing spatial awareness, achieving up to 63.3% top-1 accuracy recovery on VMamba and only 1.0% accuracy drop on PlainMamba under training-free settings.
Mamba demonstrates strong efficiency in modeling long visual sequences. However, when token reduction is applied to structurally enhanced Mamba variants, these models exhibit a severe performance collapse. We attribute this degradation to the spatially agnostic nature of existing reduction methods, which violate the two-dimensional structural premise required by the selective scanning mechanism. In this work, we propose STORM, a spatial-aware token reduction framework designed to maintain structural integrity throughout the compression process. STORM reformulates reduction into a structured operation on spatial units, enforcing localized constraints to maintain both grid topology and neighborhood coherence. As a plug-and-play module, STORM equips existing reduction pipelines with explicit spatial awareness without any training. Empirical results demonstrate that STORM achieves state-of-the-art pruning accuracy across diverse vision Mamba backbones under training-free settings. Notably, STORM delivers a substantial accuracy recovery on VMamba, outperforming prior methods by up to 63.3\% in top-1 accuracy. Meanwhile, STORM incurs only a 1.0\% accuracy drop on PlainMamba, achieving performance comparable to ViT.