A Realism Metric for Generated LiDAR Point Clouds
This addresses a key evaluation challenge for researchers and practitioners in autonomous driving and robotics using synthetic LiDAR data, though it is incremental as it builds on existing classification-based methods.
The paper tackles the problem of evaluating the realism of generated LiDAR point clouds and its impact on downstream perception, presenting a novel metric that quantifies realism and showing it correlates with segmentation performance.
A considerable amount of research is concerned with the generation of realistic sensor data. LiDAR point clouds are generated by complex simulations or learned generative models. The generated data is usually exploited to enable or improve downstream perception algorithms. Two major questions arise from these procedures: First, how to evaluate the realism of the generated data? Second, does more realistic data also lead to better perception performance? This paper addresses both questions and presents a novel metric to quantify the realism of LiDAR point clouds. Relevant features are learned from real-world and synthetic point clouds by training on a proxy classification task. In a series of experiments, we demonstrate the application of our metric to determine the realism of generated LiDAR data and compare the realism estimation of our metric to the performance of a segmentation model. We confirm that our metric provides an indication for the downstream segmentation performance.