Policy Optimization with Advantage Regularization for Long-Term Fairness in Decision Systems
This work addresses fairness in high-stake decision-making contexts like loan approval and vaccine distribution, but it is incremental as it builds on existing MDP formulations with new regularization methods.
The paper tackled the problem of achieving long-term fairness in decision systems by using policy optimization with advantage regularization, resulting in policies that often achieve higher overall utility and less fairness violation compared to previous strategies.
Long-term fairness is an important factor of consideration in designing and deploying learning-based decision systems in high-stake decision-making contexts. Recent work has proposed the use of Markov Decision Processes (MDPs) to formulate decision-making with long-term fairness requirements in dynamically changing environments, and demonstrated major challenges in directly deploying heuristic and rule-based policies that worked well in static environments. We show that policy optimization methods from deep reinforcement learning can be used to find strictly better decision policies that can often achieve both higher overall utility and less violation of the fairness requirements, compared to previously-known strategies. In particular, we propose new methods for imposing fairness requirements in policy optimization by regularizing the advantage evaluation of different actions. Our proposed methods make it easy to impose fairness constraints without reward engineering or sacrificing training efficiency. We perform detailed analyses in three established case studies, including attention allocation in incident monitoring, bank loan approval, and vaccine distribution in population networks.