RS-Corrector: Correcting the Racial Stereotypes in Latent Diffusion Models
This addresses a social bias issue in AI-generated imagery, which is an incremental improvement for mitigating stereotypes in latent diffusion models.
The paper tackles the problem of racial stereotypes in text-to-image generation models like Stable Diffusion, which produce biased images for certain racial groups. The proposed RS-Corrector framework corrects these stereotypes during inference without fine-tuning, as shown through empirical evaluations.
Recent text-conditioned image generation models have demonstrated an exceptional capacity to produce diverse and creative imagery with high visual quality. However, when pre-trained on billion-sized datasets randomly collected from the Internet, where potential biased human preferences exist, these models tend to produce images with common and recurring stereotypes, particularly for certain racial groups. In this paper, we conduct an initial analysis of the publicly available Stable Diffusion model and its derivatives, highlighting the presence of racial stereotypes. These models often generate distorted or biased images for certain racial groups, emphasizing stereotypical characteristics. To address these issues, we propose a framework called "RS-Corrector", designed to establish an anti-stereotypical preference in the latent space and update the latent code for refined generated results. The correction process occurs during the inference stage without requiring fine-tuning of the original model. Extensive empirical evaluations demonstrate that the introduced \themodel effectively corrects the racial stereotypes of the well-trained Stable Diffusion model while leaving the original model unchanged.