HRFuser: A Multi-resolution Sensor Fusion Architecture for 2D Object Detection
This work addresses the need for a generic and modular sensor fusion architecture for autonomous vehicles, offering incremental improvements over existing methods.
The authors tackled the problem of 2D object detection in autonomous vehicles by proposing HRFuser, a modular multi-resolution sensor fusion architecture that fuses multiple sensors like cameras, lidars, and radars, resulting in substantial improvements over camera-only performance and consistently outperforming state-of-the-art fusion methods on datasets such as nuScenes and DENSE.
Besides standard cameras, autonomous vehicles typically include multiple additional sensors, such as lidars and radars, which help acquire richer information for perceiving the content of the driving scene. While several recent works focus on fusing certain pairs of sensors - such as camera with lidar or radar - by using architectural components specific to the examined setting, a generic and modular sensor fusion architecture is missing from the literature. In this work, we propose HRFuser, a modular architecture for multi-modal 2D object detection. It fuses multiple sensors in a multi-resolution fashion and scales to an arbitrary number of input modalities. The design of HRFuser is based on state-of-the-art high-resolution networks for image-only dense prediction and incorporates a novel multi-window cross-attention block as the means to perform fusion of multiple modalities at multiple resolutions. We demonstrate via extensive experiments on nuScenes and the adverse conditions DENSE datasets that our model effectively leverages complementary features from additional modalities, substantially improving upon camera-only performance and consistently outperforming state-of-the-art 3D and 2D fusion methods evaluated on 2D object detection metrics. The source code is publicly available.