Introducing a Class-Aware Metric for Monocular Depth Estimation: An Automotive Perspective
This work addresses the need for more informative evaluation metrics in monocular depth estimation, particularly for safety-critical automotive scenarios, though it is incremental as it builds on existing evaluation frameworks.
The authors tackled the problem of evaluating monocular depth estimation models for automotive applications by introducing a class-aware metric that incorporates class-wise, edge/corner feature, and global consistency components, weighted by distance and criticality. The results demonstrate that this metric offers deeper insights into model performance and meets safety-critical requirements.
The increasing accuracy reports of metric monocular depth estimation models lead to a growing interest from the automotive domain. Current model evaluations do not provide deeper insights into the models' performance, also in relation to safety-critical or unseen classes. Within this paper, we present a novel approach for the evaluation of depth estimation models. Our proposed metric leverages three components, a class-wise component, an edge and corner image feature component, and a global consistency retaining component. Classes are further weighted on their distance in the scene and on criticality for automotive applications. In the evaluation, we present the benefits of our metric through comparison to classical metrics, class-wise analytics, and the retrieval of critical situations. The results show that our metric provides deeper insights into model results while fulfilling safety-critical requirements. We release the code and weights on the following repository: https://github.com/leisemann/ca_mmde