(When) Are Contrastive Explanations of Reinforcement Learning Helpful?
This addresses the challenge of making RL agents safer to deploy by evaluating explanation methods for practitioners, but it is incremental as it focuses on comparing existing explanation types.
The study investigated whether contrastive explanations improve understanding of reinforcement learning policies compared to complete explanations, finding that complete explanations are generally more effective when of equal or smaller size and not worse when larger.
Global explanations of a reinforcement learning (RL) agent's expected behavior can make it safer to deploy. However, such explanations are often difficult to understand because of the complicated nature of many RL policies. Effective human explanations are often contrastive, referencing a known contrast (policy) to reduce redundancy. At the same time, these explanations also require the additional effort of referencing that contrast when evaluating an explanation. We conduct a user study to understand whether and when contrastive explanations might be preferable to complete explanations that do not require referencing a contrast. We find that complete explanations are generally more effective when they are the same size or smaller than a contrastive explanation of the same policy, and no worse when they are larger. This suggests that contrastive explanations are not sufficient to solve the problem of effectively explaining reinforcement learning policies, and require additional careful study for use in this context.