On Interactive Explanations as Non-Monotonic Reasoning
This work addresses the issue of unreliable explanations for users interacting with AI systems, though it is incremental as it builds on existing formal reasoning concepts.
The paper tackles the problem of inconsistent local explanations across instances in interactive AI systems, which can reduce user confidence, by proposing a formal model that treats explanations as non-monotonic reasoning objects to resolve these inconsistencies and analyze interactive scenarios.
Recent work shows issues of consistency with explanations, with methods generating local explanations that seem reasonable instance-wise, but that are inconsistent across instances. This suggests not only that instance-wise explanations can be unreliable, but mainly that, when interacting with a system via multiple inputs, a user may actually lose confidence in the system. To better analyse this issue, in this work we treat explanations as objects that can be subject to reasoning and present a formal model of the interactive scenario between user and system, via sequences of inputs, outputs, and explanations. We argue that explanations can be thought of as committing to some model behaviour (even if only prima facie), suggesting a form of entailment, which, we argue, should be thought of as non-monotonic. This allows: 1) to solve some considered inconsistencies in explanation, such as via a specificity relation; 2) to consider properties from the non-monotonic reasoning literature and discuss their desirability, gaining more insight on the interactive explanation scenario.