Fair Text-to-Image Diffusion via Fair Mapping
This addresses fairness issues in AI-generated images for users of text-to-image systems, though it is incremental as it builds on existing diffusion models.
The paper tackles the problem of demographic bias in text-to-image diffusion models when generating human-related images, proposing Fair Mapping to modify pre-trained models for fairer outputs while maintaining image quality, with experiments showing significant fairness improvements.
In this paper, we address the limitations of existing text-to-image diffusion models in generating demographically fair results when given human-related descriptions. These models often struggle to disentangle the target language context from sociocultural biases, resulting in biased image generation. To overcome this challenge, we propose Fair Mapping, a flexible, model-agnostic, and lightweight approach that modifies a pre-trained text-to-image diffusion model by controlling the prompt to achieve fair image generation. One key advantage of our approach is its high efficiency. It only requires updating an additional linear network with few parameters at a low computational cost. By developing a linear network that maps conditioning embeddings into a debiased space, we enable the generation of relatively balanced demographic results based on the specified text condition. With comprehensive experiments on face image generation, we show that our method significantly improves image generation fairness with almost the same image quality compared to conventional diffusion models when prompted with descriptions related to humans. By effectively addressing the issue of implicit language bias, our method produces more fair and diverse image outputs.