Balancing Competing Objectives with Noisy Data: Score-Based Classifiers for Welfare-Aware Machine Learning
This work addresses the challenge of making welfare-aware decisions in noisy or data-limited regimes, which is incremental as it builds on existing multi-objective optimization and fairness concepts.
The paper tackles the problem of algorithmic decisions balancing private and public objectives, such as profit and social welfare, by analyzing score-based classifiers that trace an empirical Pareto frontier. It provides theoretical bounds on Pareto errors due to noisy data and shows equivalence to fairness-constrained policies, with empirical validation in content recommendation and fisheries.
While real-world decisions involve many competing objectives, algorithmic decisions are often evaluated with a single objective function. In this paper, we study algorithmic policies which explicitly trade off between a private objective (such as profit) and a public objective (such as social welfare). We analyze a natural class of policies which trace an empirical Pareto frontier based on learned scores, and focus on how such decisions can be made in noisy or data-limited regimes. Our theoretical results characterize the optimal strategies in this class, bound the Pareto errors due to inaccuracies in the scores, and show an equivalence between optimal strategies and a rich class of fairness-constrained profit-maximizing policies. We then present empirical results in two different contexts -- online content recommendation and sustainable abalone fisheries -- to underscore the applicability of our approach to a wide range of practical decisions. Taken together, these results shed light on inherent trade-offs in using machine learning for decisions that impact social welfare.