Investigating Bias in Image Classification using Model Explanations
This research addresses the problem of efficiently detecting bias in image classification for practitioners and researchers, offering insights into the utility and limitations of model explanations for this task.
This paper investigates the effectiveness of model explanations in detecting bias in image classification, aiming to remove the reliance on sensitive attributes for fairness calculations. It identifies strengths and best practices for using explanations for bias detection, but also three main weaknesses: poor estimation of bias degree, potential introduction of additional bias, and inefficiency in human effort.
We evaluated whether model explanations could efficiently detect bias in image classification by highlighting discriminating features, thereby removing the reliance on sensitive attributes for fairness calculations. To this end, we formulated important characteristics for bias detection and observed how explanations change as the degree of bias in models change. The paper identifies strengths and best practices for detecting bias using explanations, as well as three main weaknesses: explanations poorly estimate the degree of bias, could potentially introduce additional bias into the analysis, and are sometimes inefficient in terms of human effort involved.