HEX: Human-in-the-loop Explainability via Deep Reinforcement Learning
This work addresses the need for reliable, human-centric explanations in critical ML applications, though it appears incremental by building on existing MLX methods with a specific focus on decision boundaries and data constraints.
The authors tackled the problem of machine learning explainability (MLX) in high-stakes decision-making by proposing HEX, a human-in-the-loop deep reinforcement learning approach that synthesizes explanation-providing policies from any classification model, focusing on the model's decision boundary and operating in limited data scenarios like federated learning.
The use of machine learning (ML) models in decision-making contexts, particularly those used in high-stakes decision-making, are fraught with issue and peril since a person - not a machine - must ultimately be held accountable for the consequences of the decisions made using such systems. Machine learning explainability (MLX) promises to provide decision-makers with prediction-specific rationale, assuring them that the model-elicited predictions are made for the right reasons and are thus reliable. Few works explicitly consider this key human-in-the-loop (HITL) component, however. In this work we propose HEX, a human-in-the-loop deep reinforcement learning approach to MLX. HEX incorporates 0-distrust projection to synthesize decider specific explanation-providing policies from any arbitrary classification model. HEX is also constructed to operate in limited or reduced training data scenarios, such as those employing federated learning. Our formulation explicitly considers the decision boundary of the ML model in question, rather than the underlying training data, which is a shortcoming of many model-agnostic MLX methods. Our proposed methods thus synthesize HITL MLX policies that explicitly capture the decision boundary of the model in question for use in limited data scenarios.