Tradeoff-Focused Contrastive Explanation for MDP Planning
This addresses trust issues for end-users of automated planning agents in real-world applications, though it is incremental as it builds on existing contrastive explanation methods.
The paper tackles the problem of explaining complex tradeoffs in multi-objective MDP planning to improve user trust, and shows through a human experiment that their contrastive explanation approach significantly enhances users' understanding and confidence.
End-users' trust in automated agents is important as automated decision-making and planning is increasingly used in many aspects of people's lives. In real-world applications of planning, multiple optimization objectives are often involved. Thus, planning agents' decisions can involve complex tradeoffs among competing objectives. It can be difficult for the end-users to understand why an agent decides on a particular planning solution on the basis of its objective values. As a result, the users may not know whether the agent is making the right decisions, and may lack trust in it. In this work, we contribute an approach, based on contrastive explanation, that enables a multi-objective MDP planning agent to explain its decisions in a way that communicates its tradeoff rationale in terms of the domain-level concepts. We conduct a human subjects experiment to evaluate the effectiveness of our explanation approach in a mobile robot navigation domain. The results show that our approach significantly improves the users' understanding, and confidence in their understanding, of the tradeoff rationale of the planning agent.