Person-Centric Annotations of LAION-400M: Auditing Bias and Its Transfer to Models
This work addresses bias auditing in AI for researchers and practitioners by providing empirical links between dataset composition and model bias, though it is incremental as it builds on existing annotation and bias analysis methods.
The study tackled the problem of demographic biases in vision-language models by creating person-centric annotations for the LAION-400M dataset, revealing imbalances like disproportionate linking of certain groups with negative content and showing that 60-70% of gender bias in models like CLIP and Stable Diffusion can be explained by data co-occurrences.
Vision-language models trained on large-scale multimodal datasets show strong demographic biases, but the role of training data in producing these biases remains unclear. A major barrier has been the lack of demographic annotations in web-scale datasets such as LAION-400M. We address this gap by creating person-centric annotations for the full dataset, including over 276 million bounding boxes, perceived gender and race/ethnicity labels, and automatically generated captions. These annotations are produced through validated automatic labeling pipelines combining object detection, multimodal captioning, and finetuned classifiers. Using them, we uncover demographic imbalances and harmful associations, such as the disproportionate linking of men and individuals perceived as Black or Middle Eastern with crime-related and negative content. We also show that 60-70% of gender bias in CLIP and Stable Diffusion can be linearly explained by direct co-occurrences in the data. Our resources establish the first large-scale empirical link between dataset composition and downstream model bias.