Model Mis-specification and Algorithmic Bias
This work addresses bias in critical decision-making algorithms, providing a foundational step to decouple causes of bias, though it is incremental in analyzing mis-specification effects.
The paper tackles the problem of algorithmic bias in machine learning by showing that model mis-specification can lead to large, unequal prediction errors across demographic groups even with unbiased input data, with errors potentially having opposite signs for groups of equal size.
Machine learning algorithms are increasingly used to inform critical decisions. There is a growing concern about bias, that algorithms may produce uneven outcomes for individuals in different demographic groups. In this work, we measure bias as the difference between mean prediction errors across groups. We show that even with unbiased input data, when a model is mis-specified: (1) population-level mean prediction error can still be negligible, but group-level mean prediction errors can be large; (2) such errors are not equal across groups; and (3) the difference between errors, i.e., bias, can take the worst-case realization. That is, when there are two groups of the same size, mean prediction errors for these two groups have the same magnitude but opposite signs. In closed form, we show such errors and bias are functions of the first and second moments of the joint distribution of features (for linear and probit regressions). We also conduct numerical experiments to show similar results in more general settings. Our work provides a first step for decoupling the impact of different causes of bias.