Effort-Based Criticality Metrics for Evaluating 3D Perception Errors in Autonomous Driving
This addresses the need for better safety evaluation in autonomous driving by distinguishing critical perception errors, though it is incremental as it builds on prior kinematics and reachability methods.
The paper tackled the problem of evaluating 3D perception errors in autonomous driving by proposing effort-based metrics like False Speed Reduction and Maximum Deceleration Rate to quantify safety impacts, showing that 65-93% of errors are non-critical and the metrics capture safety-relevant information inaccessible to existing measures.
Criticality metrics such as time-to-collision (TTC) quantify collision urgency but conflate the consequences of false-positive (FP) and false-negative (FN) perception errors. We propose two novel effort-based metrics: False Speed Reduction (FSR), the cumulative velocity loss from persistent phantom detections, and Maximum Deceleration Rate (MDR), the peak braking demand from missed objects under a constant-acceleration model. These longitudinal metrics are complemented by Lateral Evasion Acceleration (LEA), adapted from prior lateral evasion kinematics and coupled with reachability-based collision timing to quantify the minimum steering effort to avoid a predicted collision. A reachability-based ellipsoidal collision filter ensures only dynamically plausible threats are scored, with frame-level matching and track-level aggregation. Evaluation of different perception pipelines on nuScenes and Argoverse~2 shows that 65-93% of errors are non-critical, and Spearman correlation analysis confirms that all three metrics capture safety-relevant information inaccessible to established time-based, deceleration-based, or normalized criticality measures, enabling targeted mining of the most critical perception failures.