Multi-Axis Max@K Reinforcement Learning for Representative Diversity in Text-to-Image Generation
For practitioners of text-to-image generation, this work addresses demographic bias and lack of diversity in generated images, offering a practical RL-based method to improve fairness.
Text-to-image models often generate limited diversity for a given prompt, amplifying demographic skew. The authors propose multi-axis max@K, a reinforcement learning objective that improves target-mode coverage, achieving a 0.23-0.36 Fairness Score improvement over the base model while maintaining quality.
Text-to-image (T2I) models can synthesize realistic, prompt-aligned images, yet samples generated for the same prompt often cover only a small subset of visually distinct modes. This limits the diversity of images, and for person-centric prompts, can reflect or amplify demographic skew. We formalize this problem as coverage of a predefined set of semantically specified modes, which we call target-mode coverage. We then propose multi-axis max@K, a group-based reinforcement learning objective for improving such coverage in diffusion-based T2I models. Given a group of samples and one score per target category, multi-axis max@K first takes the maximum score across samples for each category and then sums these category-wise maxima. The resulting credit assignment gives a sample positive weight on a category only when it increases that category's group-wise maximum, allowing different samples to contribute to different categories. We first validate the credit-assignment mechanism on a synthetic mixture and on SD3.5-M using deterministic pixel-based color rewards. We then evaluate the same objective on perceived-appearance fairness. Across three automatic evaluators on held-out prompts, multi-axis max@K improves the Fairness Score by 0.23-0.36 relative to the base model, while maintaining image quality and text alignment.