Learning Multi-dimensional Human Preference for Text-to-Image Generation
This addresses the need for more nuanced evaluation of text-to-image generation models, though it is incremental as it builds on existing CLIP-based methods.
The paper tackles the problem that current metrics for text-to-image models inadequately represent human preferences by reducing them to a single score, and proposes the Multi-dimensional Preference Score (MPS) model, which outperforms existing scoring methods across 3 datasets in 4 dimensions.
Current metrics for text-to-image models typically rely on statistical metrics which inadequately represent the real preference of humans. Although recent work attempts to learn these preferences via human annotated images, they reduce the rich tapestry of human preference to a single overall score. However, the preference results vary when humans evaluate images with different aspects. Therefore, to learn the multi-dimensional human preferences, we propose the Multi-dimensional Preference Score (MPS), the first multi-dimensional preference scoring model for the evaluation of text-to-image models. The MPS introduces the preference condition module upon CLIP model to learn these diverse preferences. It is trained based on our Multi-dimensional Human Preference (MHP) Dataset, which comprises 918,315 human preference choices across four dimensions (i.e., aesthetics, semantic alignment, detail quality and overall assessment) on 607,541 images. The images are generated by a wide range of latest text-to-image models. The MPS outperforms existing scoring methods across 3 datasets in 4 dimensions, enabling it a promising metric for evaluating and improving text-to-image generation.