Counterfactual Instances Explain Little
This addresses the problem of inadequate interpretability in AI for users needing reliable explanations, though it is incremental by building on existing philosophical and causal reasoning frameworks.
The paper argues that counterfactual instance explanations alone are insufficient for explaining machine learning decisions, proposing that satisfactory explanations must combine counterfactual instances with causal equations to effectively interpret predictions.
In many applications, it is important to be able to explain the decisions of machine learning systems. An increasingly popular approach has been to seek to provide \emph{counterfactual instance explanations}. These specify close possible worlds in which, contrary to the facts, a person receives their desired decision from the machine learning system. This paper will draw on literature from the philosophy of science to argue that a satisfactory explanation must consist of both counterfactual instances and a causal equation (or system of equations) that support the counterfactual instances. We will show that counterfactual instances by themselves explain little. We will further illustrate how explainable AI methods that provide both causal equations and counterfactual instances can successfully explain machine learning predictions.