Transferring Fairness using Multi-Task Learning with Limited Demographic Information
This work addresses fairness in ML for applications with limited demographic data, offering a transfer learning solution that is incremental in adapting existing fairness losses to multi-task settings.
The paper tackles the problem of improving fairness in machine learning predictions across demographic groups when demographic annotations are scarce, by proposing a multi-task learning approach that transfers fairness objectives from a related task with demographic data to a target task without such labels, demonstrating that this method can enhance fairness and enable intersectional fairness across different datasets.
Training supervised machine learning systems with a fairness loss can improve prediction fairness across different demographic groups. However, doing so requires demographic annotations for training data, without which we cannot produce debiased classifiers for most tasks. Drawing inspiration from transfer learning methods, we investigate whether we can utilize demographic data from a related task to improve the fairness of a target task. We adapt a single-task fairness loss to a multi-task setting to exploit demographic labels from a related task in debiasing a target task and demonstrate that demographic fairness objectives transfer fairness within a multi-task framework. Additionally, we show that this approach enables intersectional fairness by transferring between two datasets with different single-axis demographics. We explore different data domains to show how our loss can improve fairness domains and tasks.