BXRL: Behavior-Explainable Reinforcement Learning
This work addresses a foundational gap in XRL for researchers and practitioners by formalizing behavior explanation, but it is incremental as it builds on existing methods without new empirical results.
The paper tackles the lack of a formal definition for behavior in Explainable Reinforcement Learning (XRL) by introducing Behavior-Explainable Reinforcement Learning (BXRL), which treats behaviors as first-class objects and enables queries to explain specific behaviors, though no implementation or concrete performance numbers are provided.
A major challenge of Reinforcement Learning is that agents often learn undesired behaviors that seem to defy the reward structure they were given. Explainable Reinforcement Learning (XRL) methods can answer queries such as "explain this specific action", "explain this specific trajectory", and "explain the entire policy". However, XRL lacks a formal definition for behavior as a pattern of actions across many episodes. We provide such a definition, and use it to enable a new query: "Explain this behavior". We present Behavior-Explainable Reinforcement Learning (BXRL), a new problem formulation that treats behaviors as first-class objects. BXRL defines a behavior measure as any function $m : Î \to \mathbb{R}$, allowing users to precisely express the pattern of actions that they find interesting and measure how strongly the policy exhibits it. We define contrastive behaviors that reduce the question "why does the agent prefer $a$ to $a'$?" to "why is $m(Ï)$ high?" which can be explored with differentiation. We do not implement an explainability method; we instead analyze three existing methods and propose how they could be adapted to explain behavior. We present a port of the HighwayEnv driving environment to JAX, which provides an interface for defining, measuring, and differentiating behaviors with respect to the model parameters.