Extracting Incentives from Black-Box Decisions
This addresses the need for clarity on algorithmic incentives for decision-makers and recipients, with potential impacts in fairness and transparency, though it is incremental in applying existing MDP toolkits to a new problem.
The paper tackles the problem of understanding which actions individuals are incentivized to take to improve algorithmic decisions, particularly for non-linear models like neural networks, by proposing a framework that models this as a Markov Decision Process and demonstrates its utility in real-world settings such as recidivism prediction and credit scoring.
An algorithmic decision-maker incentivizes people to act in certain ways to receive better decisions. These incentives can dramatically influence subjects' behaviors and lives, and it is important that both decision-makers and decision-recipients have clarity on which actions are incentivized by the chosen model. While for linear functions, the changes a subject is incentivized to make may be clear, we prove that for many non-linear functions (e.g. neural networks, random forests), classical methods for interpreting the behavior of models (e.g. input gradients) provide poor advice to individuals on which actions they should take. In this work, we propose a mathematical framework for understanding algorithmic incentives as the challenge of solving a Markov Decision Process, where the state includes the set of input features, and the reward is a function of the model's output. We can then leverage the many toolkits for solving MDPs (e.g. tree-based planning, reinforcement learning) to identify the optimal actions each individual is incentivized to take to improve their decision under a given model. We demonstrate the utility of our method by estimating the maximally-incentivized actions in two real-world settings: a recidivism risk predictor we train using ProPublica's COMPAS dataset, and an online credit scoring tool published by the Fair Isaac Corporation (FICO).