Temporal Counterfactual Explanations of Behaviour Tree Decisions
For users of behaviour tree-driven robots, this provides a novel way to obtain causal, counterfactual explanations of robot decisions, enhancing transparency and trust.
The paper introduces a method to automatically generate counterfactual explanations for decisions made by behaviour tree-driven robots, enabling real-time causal explanations for contrastive 'why' questions. The approach builds a causal model from the behaviour tree structure and domain knowledge, then queries it to find diverse counterfactuals, outperforming prior methods that lack causal or consistent explanations.
Explainability, in particular, the ability for robots to explain why they have made a decision or behaved in a certain way, is a critical tool in helping users understand the robots they interact and coexist with. Behaviour trees are a popular framework for controlling the decision-making of robots, and thus a natural question to ask is whether or not a system driven by a behaviour tree is capable of answering "why" questions. While explainability for behaviour tree-driven robots has seen some prior attention, no existing methods are capable of generating causal, counterfactual explanations which detail the reasons for robot decisions and behaviour. Therefore, in this work, we introduce a novel approach which automatically generates counterfactual explanations in response to contrastive "why" questions. Our method achieves this by first automatically building a causal model from the structure of the behaviour tree as well as domain knowledge about the state and individual behaviour tree nodes. The resultant causal model is then queried and searched to find a set of diverse counterfactual explanations. We demonstrate that our approach is able to correctly explain the behaviour of a wide range of behaviour tree structures and states in real time, unlike previous methods which are either unable to answer contrastive questions with causal explanations, or are not guaranteed to provide consistent and accurate explanations. By being able to answer a wide range of causal queries, our approach represents a step towards more transparent, understandable, and ultimately safe and trustworthy robotic systems.