Risk Ranked Recall: Collision Safety Metric for Object Detection Systems in Autonomous Vehicles
This addresses safety evaluation for autonomous vehicles, but appears incremental as it modifies existing recall metrics.
The authors tackled the problem that standard object detection metrics do not adequately assess safety for autonomous vehicles, and introduced Risk Ranked Recall (R^3) metrics that categorize objects by collision risk and measure recall per rank.
Commonly used metrics for evaluation of object detection systems (precision, recall, mAP) do not give complete information about their suitability of use in safety critical tasks, like obstacle detection for collision avoidance in Autonomous Vehicles (AV). This work introduces the Risk Ranked Recall ($R^3$) metrics for object detection systems. The $R^3$ metrics categorize objects within three ranks. Ranks are assigned based on an objective cyber-physical model for the risk of collision. Recall is measured for each rank.