NeuRadar: Neural Radiance Fields for Automotive Radar Point Clouds
This work addresses the need for efficient testing and validation in autonomous driving by enabling simulation of radar data, which is robust in adverse conditions, though it is incremental as it adapts existing NeRF methods to radar.
The authors tackled the problem of novel view synthesis for automotive radar point clouds, which was previously unexplored, by developing NeuRadar, a NeRF-based model that jointly generates radar point clouds, camera images, and lidar point clouds, achieving realistic reconstruction results on two automotive datasets and establishing a baseline for such models.
Radar is an important sensor for autonomous driving (AD) systems due to its robustness to adverse weather and different lighting conditions. Novel view synthesis using neural radiance fields (NeRFs) has recently received considerable attention in AD due to its potential to enable efficient testing and validation but remains unexplored for radar point clouds. In this paper, we present NeuRadar, a NeRF-based model that jointly generates radar point clouds, camera images, and lidar point clouds. We explore set-based object detection methods such as DETR, and propose an encoder-based solution grounded in the NeRF geometry for improved generalizability. We propose both a deterministic and a probabilistic point cloud representation to accurately model the radar behavior, with the latter being able to capture radar's stochastic behavior. We achieve realistic reconstruction results for two automotive datasets, establishing a baseline for NeRF-based radar point cloud simulation models. In addition, we release radar data for ZOD's Sequences and Drives to enable further research in this field. To encourage further development of radar NeRFs, we release the source code for NeuRadar.