Navigating Ensemble Configurations for Algorithmic Fairness
This work addresses the challenge for practitioners in machine learning to navigate trade-offs between fairness and performance when using ensembles with bias mitigators, though it is incremental as it builds on existing methods.
The researchers tackled the instability of fairness mitigators across data splits by exploring how to combine them with ensemble methods, resulting in a guidance diagram for practitioners that is robust and reproducible based on empirical tests across 13 datasets.
Bias mitigators can improve algorithmic fairness in machine learning models, but their effect on fairness is often not stable across data splits. A popular approach to train more stable models is ensemble learning, but unfortunately, it is unclear how to combine ensembles with mitigators to best navigate trade-offs between fairness and predictive performance. To that end, we built an open-source library enabling the modular composition of 8 mitigators, 4 ensembles, and their corresponding hyperparameters, and we empirically explored the space of configurations on 13 datasets. We distilled our insights from this exploration in the form of a guidance diagram for practitioners that we demonstrate is robust and reproducible.