Driving through the Concept Gridlock: Unraveling Explainability Bottlenecks in Automated Driving
This work addresses the need for user acceptance and understanding of decisions in human-assisted or autonomous driving systems, though it is incremental as it builds on existing concept bottleneck methods.
The paper tackled the problem of explainability in automated driving by using concept bottleneck models to interpret sequential driving scenes and predict control commands, achieving competitive performance compared to latent visual features while maintaining interpretability.
Concept bottleneck models have been successfully used for explainable machine learning by encoding information within the model with a set of human-defined concepts. In the context of human-assisted or autonomous driving, explainability models can help user acceptance and understanding of decisions made by the autonomous vehicle, which can be used to rationalize and explain driver or vehicle behavior. We propose a new approach using concept bottlenecks as visual features for control command predictions and explanations of user and vehicle behavior. We learn a human-understandable concept layer that we use to explain sequential driving scenes while learning vehicle control commands. This approach can then be used to determine whether a change in a preferred gap or steering commands from a human (or autonomous vehicle) is led by an external stimulus or change in preferences. We achieve competitive performance to latent visual features while gaining interpretability within our model setup.