Revisiting Evaluation of Deep Neural Networks for Pedestrian Detection
This work addresses the need for more robust and safety-critical evaluation methods in pedestrian detection for automated driving systems, representing an incremental improvement.
The authors tackled the problem of unreliable performance benchmarks for pedestrian detection in automated driving by proposing eight error categories and new metrics for fine-grained evaluation, achieving state-of-the-art results on CityPersons-reasonable without extra training data.
Reliable pedestrian detection represents a crucial step towards automated driving systems. However, the current performance benchmarks exhibit weaknesses. The currently applied metrics for various subsets of a validation dataset prohibit a realistic performance evaluation of a DNN for pedestrian detection. As image segmentation supplies fine-grained information about a street scene, it can serve as a starting point to automatically distinguish between different types of errors during the evaluation of a pedestrian detector. In this work, eight different error categories for pedestrian detection are proposed and new metrics are proposed for performance comparison along these error categories. We use the new metrics to compare various backbones for a simplified version of the APD, and show a more fine-grained and robust way to compare models with each other especially in terms of safety-critical performance. We achieve SOTA on CityPersons-reasonable (without extra training data) by using a rather simple architecture.