BAANet: Learning Bi-directional Adaptive Attention Gates for Multispectral Pedestrian Detection
This work addresses pedestrian detection in challenging conditions like low illumination for applications such as autonomous driving, but it is incremental as it builds on existing attention-based fusion methods.
The paper tackles multispectral pedestrian detection by proposing a cross-modality fusion module called Bi-directional Adaptive Attention Gate (BAA-Gate) to address noise in RGB and thermal infrared features, achieving superior performance on the KAIST dataset with satisfactory speed.
Thermal infrared (TIR) image has proven effectiveness in providing temperature cues to the RGB features for multispectral pedestrian detection. Most existing methods directly inject the TIR modality into the RGB-based framework or simply ensemble the results of two modalities. This, however, could lead to inferior detection performance, as the RGB and TIR features generally have modality-specific noise, which might worsen the features along with the propagation of the network. Therefore, this work proposes an effective and efficient cross-modality fusion module called Bi-directional Adaptive Attention Gate (BAA-Gate). Based on the attention mechanism, the BAA-Gate is devised to distill the informative features and recalibrate the representations asymptotically. Concretely, a bi-direction multi-stage fusion strategy is adopted to progressively optimize features of two modalities and retain their specificity during the propagation. Moreover, an adaptive interaction of BAA-Gate is introduced by the illumination-based weighting strategy to adaptively adjust the recalibrating and aggregating strength in the BAA-Gate and enhance the robustness towards illumination changes. Considerable experiments on the challenging KAIST dataset demonstrate the superior performance of our method with satisfactory speed.