Prediction without Preclusion: Recourse Verification with Reachable Sets
This addresses the problem of unfair model predictions in high-stakes domains like finance and employment, though it is incremental as it builds on existing recourse methods.
The authors tackled the problem of machine learning models assigning fixed predictions that preclude individuals from accessing loans or jobs, by introducing recourse verification with reachable sets to test model responsiveness. Their empirical study on consumer finance datasets highlighted how models inadvertently preclude access, underscoring the need for actionability in development.
Machine learning models are often used to decide who receives a loan, a job interview, or a public benefit. Models in such settings use features without considering their actionability. As a result, they can assign predictions that are fixed $-$ meaning that individuals who are denied loans and interviews are, in fact, precluded from access to credit and employment. In this work, we introduce a procedure called recourse verification to test if a model assigns fixed predictions to its decision subjects. We propose a model-agnostic approach for recourse verification with reachable sets $-$ i.e., the set of all points that a person can reach through their actions in feature space. We develop methods to construct reachable sets for discrete feature spaces, which can certify the responsiveness of any model by simply querying its predictions. We conduct a comprehensive empirical study on the infeasibility of recourse on datasets from consumer finance. Our results highlight how models can inadvertently preclude access by assigning fixed predictions and underscore the need to account for actionability in model development.