Cross-Modal Object Tracking via Modality-Aware Fusion Network and A Large-Scale Dataset
This addresses tracking robustness in varying light conditions for surveillance applications, but it is incremental as it builds on existing multi-modal fusion approaches.
The paper tackles the problem of object tracking across RGB and near-infrared (NIR) modalities, which is challenging due to appearance gaps and lack of switch signals, by proposing the Modality-Aware Fusion Network (MAFNet) that adaptively fuses information to achieve state-of-the-art performance on a new large-scale dataset.
Visual tracking often faces challenges such as invalid targets and decreased performance in low-light conditions when relying solely on RGB image sequences. While incorporating additional modalities like depth and infrared data has proven effective, existing multi-modal imaging platforms are complex and lack real-world applicability. In contrast, near-infrared (NIR) imaging, commonly used in surveillance cameras, can switch between RGB and NIR based on light intensity. However, tracking objects across these heterogeneous modalities poses significant challenges, particularly due to the absence of modality switch signals during tracking. To address these challenges, we propose an adaptive cross-modal object tracking algorithm called Modality-Aware Fusion Network (MAFNet). MAFNet efficiently integrates information from both RGB and NIR modalities using an adaptive weighting mechanism, effectively bridging the appearance gap and enabling a modality-aware target representation. It consists of two key components: an adaptive weighting module and a modality-specific representation module......