Generating Justifications for Norm-Related Agent Decisions
This work addresses the need for explainable AI in norm-governed systems, but it is incremental as it builds on existing norm-based reasoning and natural language generation techniques.
The authors tackled the problem of generating natural language justifications for decisions made by norm-based agents, by developing a method to convert temporal logic statements into explanatory sentences, and evaluated it using human judgments on intelligibility, mental model, and trust.
We present an approach to generating natural language justifications of decisions derived from norm-based reasoning. Assuming an agent which maximally satisfies a set of rules specified in an object-oriented temporal logic, the user can ask factual questions (about the agent's rules, actions, and the extent to which the agent violated the rules) as well as "why" questions that require the agent comparing actual behavior to counterfactual trajectories with respect to these rules. To produce natural-sounding explanations, we focus on the subproblem of producing natural language clauses from statements in a fragment of temporal logic, and then describe how to embed these clauses into explanatory sentences. We use a human judgment evaluation on a testbed task to compare our approach to variants in terms of intelligibility, mental model and perceived trust.