PyCFRL: A Python library for counterfactually fair offline reinforcement learning via sequential data preprocessing
This addresses fairness issues in RL for minority or disadvantaged groups, but it appears incremental as it builds on existing fairness concepts with a new library implementation.
The authors tackled the problem of reinforcement learning policies disadvantaging minority or socioeconomically disadvantaged groups by introducing PyCFRL, a Python library that implements a novel data preprocessing algorithm for ensuring counterfactual fairness in offline RL, with the library publicly available on PyPI and GitHub.
Reinforcement learning (RL) aims to learn and evaluate a sequential decision rule, often referred to as a "policy", that maximizes the population-level benefit in an environment across possibly infinitely many time steps. However, the sequential decisions made by an RL algorithm, while optimized to maximize overall population benefits, may disadvantage certain individuals who are in minority or socioeconomically disadvantaged groups. To address this problem, we introduce PyCFRL, a Python library for ensuring counterfactual fairness in offline RL. PyCFRL implements a novel data preprocessing algorithm for learning counterfactually fair RL policies from offline datasets and provides tools to evaluate the values and counterfactual unfairness levels of RL policies. We describe the high-level functionalities of PyCFRL and demonstrate one of its major use cases through a data example. The library is publicly available on PyPI and Github (https://github.com/JianhanZhang/PyCFRL), and detailed tutorials can be found in the PyCFRL documentation (https://pycfrl-documentation.netlify.app).