MambaVLT: Time-Evolving Multimodal State Space Model for Vision-Language Tracking
This work addresses the challenge of effectively using temporal information in vision-language tracking, which is important for applications like video analysis, but it appears incremental as it builds on existing Mamba models for a specific task.
The paper tackled the problem of vision-language tracking by proposing MambaVLT, a model that uses a state space model to exploit temporal information and dynamically update reference features, achieving favorable performance against state-of-the-art trackers on diverse benchmarks.
The vision-language tracking task aims to perform object tracking based on various modality references. Existing Transformer-based vision-language tracking methods have made remarkable progress by leveraging the global modeling ability of self-attention. However, current approaches still face challenges in effectively exploiting the temporal information and dynamically updating reference features during tracking. Recently, the State Space Model (SSM), known as Mamba, has shown astonishing ability in efficient long-sequence modeling. Particularly, its state space evolving process demonstrates promising capabilities in memorizing multimodal temporal information with linear complexity. Witnessing its success, we propose a Mamba-based vision-language tracking model to exploit its state space evolving ability in temporal space for robust multimodal tracking, dubbed MambaVLT. In particular, our approach mainly integrates a time-evolving hybrid state space block and a selective locality enhancement block, to capture contextual information for multimodal modeling and adaptive reference feature update. Besides, we introduce a modality-selection module that dynamically adjusts the weighting between visual and language references, mitigating potential ambiguities from either reference type. Extensive experimental results show that our method performs favorably against state-of-the-art trackers across diverse benchmarks.