Flow-Based Feature Fusion for Vehicle-Infrastructure Cooperative 3D Object Detection
This addresses the challenge of efficient and accurate perception for autonomous driving systems by improving cooperative detection with infrastructure data, though it appears incremental as it builds on existing fusion methods.
The paper tackles the problem of uncertain temporal asynchrony and limited communication in vehicle-infrastructure cooperative 3D object detection by proposing FFNet, a flow-based feature fusion framework that predicts future features to compensate for asynchrony, resulting in outperforming existing methods while reducing transmission cost to about 1/100 of raw data on the DAIR-V2X dataset.
Cooperatively utilizing both ego-vehicle and infrastructure sensor data can significantly enhance autonomous driving perception abilities. However, the uncertain temporal asynchrony and limited communication conditions can lead to fusion misalignment and constrain the exploitation of infrastructure data. To address these issues in vehicle-infrastructure cooperative 3D (VIC3D) object detection, we propose the Feature Flow Net (FFNet), a novel cooperative detection framework. FFNet is a flow-based feature fusion framework that uses a feature flow prediction module to predict future features and compensate for asynchrony. Instead of transmitting feature maps extracted from still-images, FFNet transmits feature flow, leveraging the temporal coherence of sequential infrastructure frames. Furthermore, we introduce a self-supervised training approach that enables FFNet to generate feature flow with feature prediction ability from raw infrastructure sequences. Experimental results demonstrate that our proposed method outperforms existing cooperative detection methods while only requiring about 1/100 of the transmission cost of raw data and covers all latency in one model on the DAIR-V2X dataset. The code is available at \href{https://github.com/haibao-yu/FFNet-VIC3D}{https://github.com/haibao-yu/FFNet-VIC3D}.