Beyond Overcorrection: Evaluating Diversity in T2I Models with DivBench
This addresses the issue of inappropriate diversification in T2I models for users seeking accurate and fair image generation, though it is incremental as it builds on existing methods.
The paper tackles the problem of over-diversification in text-to-image models, where demographic attributes are inappropriately altered, and finds that context-aware methods like LLM-guided FairDiffusion can effectively balance diversity and semantic fidelity.
Current diversification strategies for text-to-image (T2I) models often ignore contextual appropriateness, leading to over-diversification where demographic attributes are modified even when explicitly specified in prompts. This paper introduces DIVBENCH, a benchmark and evaluation framework for measuring both under- and over-diversification in T2I generation. Through systematic evaluation of state-of-the-art T2I models, we find that while most models exhibit limited diversity, many diversification approaches overcorrect by inappropriately altering contextually-specified attributes. We demonstrate that context-aware methods, particularly LLM-guided FairDiffusion and prompt rewriting, can already effectively address under-diversity while avoiding over-diversification, achieving a better balance between representation and semantic fidelity.