Attention IoU: Examining Biases in CelebA using Attention Maps
This addresses bias quantification in classification models for researchers and practitioners, though it is incremental as it builds on existing bias analysis methods.
The authors tackled the problem of bias in computer vision models by introducing the Attention-IoU metric, which uses attention maps to reveal internal biases, and validated it on synthetic and CelebA datasets, uncovering correlations beyond accuracy disparities.
Computer vision models have been shown to exhibit and amplify biases across a wide array of datasets and tasks. Existing methods for quantifying bias in classification models primarily focus on dataset distribution and model performance on subgroups, overlooking the internal workings of a model. We introduce the Attention-IoU (Attention Intersection over Union) metric and related scores, which use attention maps to reveal biases within a model's internal representations and identify image features potentially causing the biases. First, we validate Attention-IoU on the synthetic Waterbirds dataset, showing that the metric accurately measures model bias. We then analyze the CelebA dataset, finding that Attention-IoU uncovers correlations beyond accuracy disparities. Through an investigation of individual attributes through the protected attribute of Male, we examine the distinct ways biases are represented in CelebA. Lastly, by subsampling the training set to change attribute correlations, we demonstrate that Attention-IoU reveals potential confounding variables not present in dataset labels.