Online Monitoring for Neural Network Based Monocular Pedestrian Pose Estimation
This addresses safety concerns in self-driving cars by providing a tool to detect and mitigate failures in real-time pose estimation, though it is incremental as it builds on existing monitoring approaches.
The paper tackles the problem of unexpected failures in neural networks for monocular pedestrian pose estimation by developing an online monitoring system, resulting in a 12.5% improvement in average error and a 126.5% improvement in worst-case error when discarding flagged outputs.
Several autonomy pipelines now have core components that rely on deep learning approaches. While these approaches work well in nominal conditions, they tend to have unexpected and severe failure modes that create concerns when used in safety-critical applications, including self-driving cars. There are several works that aim to characterize the robustness of networks offline, but currently there is a lack of tools to monitor the correctness of network outputs online during operation. We investigate the problem of online output monitoring for neural networks that estimate 3D human shapes and poses from images. Our first contribution is to present and evaluate model-based and learning-based monitors for a human-pose-and-shape reconstruction network, and assess their ability to predict the output loss for a given test input. As a second contribution, we introduce an Adversarially-Trained Online Monitor ( ATOM ) that learns how to effectively predict losses from data. ATOM dominates model-based baselines and can detect bad outputs, leading to substantial improvements in human pose output quality. Our final contribution is an extensive experimental evaluation that shows that discarding outputs flagged as incorrect by ATOM improves the average error by 12.5%, and the worst-case error by 126.5%.