Towards Trustworthy and Explainable AI for Perception Models: From Concept to Prototype Vehicle Deployment
It provides a concrete implementation of Trustworthy AI principles for 3D scene understanding in autonomous driving, addressing the gap between theoretical frameworks and practical deployment.
The paper presents a Trustworthy AI perception module for autonomous driving that integrates faithful explainability, calibrated uncertainty estimates, and enhanced robustness, validated through experiments and deployed in a prototype vehicle with a real-time XAI interface.
Deep Neural Networks have become the dominant solution for Autonomous Driving perception, but their opacity conflicts with emerging Trustworthy AI guidelines and complicates safety assurance, debugging, and human oversight. While theoretical frameworks for safe and Explainable AI (XAI) exist, concrete implementations of Trustworthy AI for 3D scene understanding remain scarce. We address this gap by proposing a Trustworthy AI perception module that is remarkably robust, integrates faithful explainability, and calibrated uncertainty estimates. Building on a transformer-based detector, we derive explanation from the attention mechanism at inference time and validate their faithfulness using perturbation-based consistency tests. We further integrate an uncertainty estimation and calibration module, and apply robustness-enhancing training methods. Experiments show faithful saliency behavior, improved robustness, and well-calibrated uncertainty estimates. Finally, we deploy these Trustworthy AI elements in a prototype vehicle and provide an XAI Interface that visualizes documentation artifacts, model uncertainty state, and saliency maps, demonstrating the feasibility of trustworthy perception monitoring in real time. Supplementary materials are available at https://tillbeemelmanns.github.io/trustworthy_ai/ .