Checklist to Define the Identification of TP, FP, and FN Object Detections in Automated Driving
This work addresses a practical problem for practitioners in automated driving by offering a tool to improve the reliability and comparability of object perception tests, though it is incremental as it builds on existing testing frameworks.
The paper tackles the lack of a comprehensive definition for identifying true positive, false-positive, and false-negative detections in automated driving systems, providing a checklist of functional aspects and implementation details to minimize ambiguity in tests.
The object perception of automated driving systems must pass quality and robustness tests before a safe deployment. Such tests typically identify true positive (TP), false-positive (FP), and false-negative (FN) detections and aggregate them to metrics. Since the literature seems to be lacking a comprehensive way to define the identification of TPs/FPs/FNs, this paper provides a checklist of relevant functional aspects and implementation details. Besides labeling policies of the test set, we cover areas of vision, occlusion handling, safety-relevant areas, matching criteria, temporal and probabilistic issues, and further aspects. Even though the checklist cannot be fully formalized, it can help practitioners minimize the ambiguity of their tests, which, in turn, makes statements on object perception more reliable and comparable.