Mitigating Inappropriateness in Image Generation: Can there be Value in Reflecting the World's Ugliness?
This addresses the issue of inappropriate content reproduction in AI-generated images for users and developers, though it is incremental as it builds on existing mitigation approaches.
The study tackled the problem of inappropriate content generation in text-to-image models by demonstrating large-scale inappropriate degeneration and evaluating mitigation strategies at inference, finding that models' representations of the world's ugliness can be used to align them with human preferences.
Text-conditioned image generation models have recently achieved astonishing results in image quality and text alignment and are consequently employed in a fast-growing number of applications. Since they are highly data-driven, relying on billion-sized datasets randomly scraped from the web, they also reproduce inappropriate human behavior. Specifically, we demonstrate inappropriate degeneration on a large-scale for various generative text-to-image models, thus motivating the need for monitoring and moderating them at deployment. To this end, we evaluate mitigation strategies at inference to suppress the generation of inappropriate content. Our findings show that we can use models' representations of the world's ugliness to align them with human preferences.