Auditing Gender Presentation Differences in Text-to-Image Models
This work addresses the issue of gender stereotypes in AI-generated images, which is important for fairness and bias mitigation in content-creation tools, though it is incremental in improving measurement techniques.
The paper tackles the problem of gender presentation differences in text-to-image models by proposing a paradigm and metric (GEP) to quantify these differences, achieving a higher correlation with human annotations than existing methods across three state-of-the-art models.
Text-to-image models, which can generate high-quality images based on textual input, have recently enabled various content-creation tools. Despite significantly affecting a wide range of downstream applications, the distributions of these generated images are still not fully understood, especially when it comes to the potential stereotypical attributes of different genders. In this work, we propose a paradigm (Gender Presentation Differences) that utilizes fine-grained self-presentation attributes to study how gender is presented differently in text-to-image models. By probing gender indicators in the input text (e.g., "a woman" or "a man"), we quantify the frequency differences of presentation-centric attributes (e.g., "a shirt" and "a dress") through human annotation and introduce a novel metric: GEP. Furthermore, we propose an automatic method to estimate such differences. The automatic GEP metric based on our approach yields a higher correlation with human annotations than that based on existing CLIP scores, consistently across three state-of-the-art text-to-image models. Finally, we demonstrate the generalization ability of our metrics in the context of gender stereotypes related to occupations.