Joint Optimization of AI Fairness and Utility: A Human-Centered Approach
This work addresses fairness-utility trade-offs in high-stakes AI applications for policymakers, offering a human-centered approach that is incremental by building on existing bias mitigation methods.
The paper tackles the problem of balancing fairness and utility in AI decision-making by proposing a framework to incorporate human policymakers' preferences into model optimization, resulting in methods that align AI behavior with specified trade-offs.
Today, AI is increasingly being used in many high-stakes decision-making applications in which fairness is an important concern. Already, there are many examples of AI being biased and making questionable and unfair decisions. The AI research community has proposed many methods to measure and mitigate unwanted biases, but few of them involve inputs from human policy makers. We argue that because different fairness criteria sometimes cannot be simultaneously satisfied, and because achieving fairness often requires sacrificing other objectives such as model accuracy, it is key to acquire and adhere to human policy makers' preferences on how to make the tradeoff among these objectives. In this paper, we propose a framework and some exemplar methods for eliciting such preferences and for optimizing an AI model according to these preferences.