Latency-aware Road Anomaly Segmentation in Videos: A Photorealistic Dataset and New Metrics
This work addresses a critical safety gap in autonomous driving by providing a realistic benchmark for anomaly segmentation, though it is incremental as it builds on existing methods with a new dataset and metrics.
The authors tackled the problem of road anomaly segmentation for autonomous driving by creating the first video dataset with 120,000 high-resolution frames at 60 FPS across 7 towns, using synthetic data enhanced for photorealism, and introduced new metrics like latency-aware streaming accuracy to measure crash prevention in real-time settings.
In the past several years, road anomaly segmentation is actively explored in the academia and drawing growing attention in the industry. The rationale behind is straightforward: if the autonomous car can brake before hitting an anomalous object, safety is promoted. However, this rationale naturally calls for a temporally informed setting while existing methods and benchmarks are designed in an unrealistic frame-wise manner. To bridge this gap, we contribute the first video anomaly segmentation dataset for autonomous driving. Since placing various anomalous objects on busy roads and annotating them in every frame are dangerous and expensive, we resort to synthetic data. To improve the relevance of this synthetic dataset to real-world applications, we train a generative adversarial network conditioned on rendering G-buffers for photorealism enhancement. Our dataset consists of 120,000 high-resolution frames at a 60 FPS framerate, as recorded in 7 different towns. As an initial benchmarking, we provide baselines using latest supervised and unsupervised road anomaly segmentation methods. Apart from conventional ones, we focus on two new metrics: temporal consistency and latencyaware streaming accuracy. We believe the latter is valuable as it measures whether an anomaly segmentation algorithm can truly prevent a car from crashing in a temporally informed setting.