Towards a Generalizable Fusion Architecture for Multimodal Object Detection
This addresses the need for robust object detection in challenging conditions like low-light or aerial scenarios, but it is incremental as it builds on existing fusion methods.
The paper tackles the problem of improving multimodal object detection by introducing FMCAF, a preprocessing architecture that enhances fusion of RGB and infrared inputs, achieving +13.9% mAP@50 on VEDAI and +1.1% on LLVIP.
Multimodal object detection improves robustness in chal- lenging conditions by leveraging complementary cues from multiple sensor modalities. We introduce Filtered Multi- Modal Cross Attention Fusion (FMCAF), a preprocess- ing architecture designed to enhance the fusion of RGB and infrared (IR) inputs. FMCAF combines a frequency- domain filtering block (Freq-Filter) to suppress redun- dant spectral features with a cross-attention-based fusion module (MCAF) to improve intermodal feature sharing. Unlike approaches tailored to specific datasets, FMCAF aims for generalizability, improving performance across different multimodal challenges without requiring dataset- specific tuning. On LLVIP (low-light pedestrian detec- tion) and VEDAI (aerial vehicle detection), FMCAF outper- forms traditional fusion (concatenation), achieving +13.9% mAP@50 on VEDAI and +1.1% on LLVIP. These results support the potential of FMCAF as a flexible foundation for robust multimodal fusion in future detection pipelines.