RecourseBench: A Modular Framework for Reproducible Algorithmic Recourse Evaluation
For researchers and practitioners in algorithmic recourse, this framework addresses the lack of standardized, reproducible evaluation, enabling fair and reliable comparison of methods.
RecourseBench is a modular, reproducible evaluation framework for algorithmic recourse methods that integrates 28 state-of-the-art methods with automated testing to ensure fidelity to original results. It provides an interactive interface for flexible comparison across methods, datasets, and models.
Algorithmic recourse methods provide counterfactual explanations that inform individuals of the actions required to overturn an unfavorable model decision. Despite rapid methodological progress, principled comparison remains elusive; existing frameworks are often difficult to extend and lack both interoperability and systematic verification that integrated methods faithfully reproduce their originally reported results. We introduce \emph{RecourseBench}, a unified evaluation framework built around three commitments namely, modularity, reproducibility, and interactivity. The framework decomposes the pipeline into five fully decoupled layers -- Data, Preprocessing, Model, Recourse Method, and Evaluation -- governed by abstract interfaces and a dynamic registry. To address the reproducibility gap in prior benchmarks, we introduce a four-tier classification system in which every integrated method is validated by an automated test suite against its originally reported results. We further provide an interactive web interface for flexible, configuration-driven comparison across methods, datasets, and model architectures. Our framework currently integrates 28 state-of-the-art recourse methods and, to our knowledge, constitutes the first recourse benchmark to explicitly enforce method-level reproducibility through automated, quantitative testing.