Explicablility as Minimizing Distance from Expected Behavior
This addresses the issue of human-AI collaboration by improving plan explicability, which is incremental as it builds on existing planning methods with a focus on human perception.
The paper tackles the problem of AI agents generating plans that are inexplicable to humans by modeling explicability as the distance between the agent's plan and the human's expected plan, and develops an anytime search algorithm to generate progressively explicable plans. It evaluates the approach in simulated autonomous car and physical robot domains, though no concrete performance numbers are provided.
In order to have effective human-AI collaboration, it is necessary to address how the AI agent's behavior is being perceived by the humans-in-the-loop. When the agent's task plans are generated without such considerations, they may often demonstrate inexplicable behavior from the human's point of view. This problem may arise due to the human's partial or inaccurate understanding of the agent's planning model. This may have serious implications from increased cognitive load to more serious concerns of safety around a physical agent. In this paper, we address this issue by modeling plan explicability as a function of the distance between a plan that agent makes and the plan that human expects it to make. We learn a regression model for mapping the plan distances to explicability scores of plans and develop an anytime search algorithm that can use this model as a heuristic to come up with progressively explicable plans. We evaluate the effectiveness of our approach in a simulated autonomous car domain and a physical robot domain.