Exploring Palette based Color Guidance in Diffusion Models
This work addresses the need for enhanced color control in AI-generated images for users in creative and design fields, representing an incremental improvement over existing methods.
The paper tackles the problem of limited color scheme control in text-to-image diffusion models, particularly for background elements and less prominent objects, by proposing palette-based color guidance. The result shows that this approach significantly improves the generation of images with desired color schemes, enabling more controlled colorization.
With the advent of diffusion models, Text-to-Image (T2I) generation has seen substantial advancements. Current T2I models allow users to specify object colors using linguistic color names, and some methods aim to personalize color-object association through prompt learning. However, existing models struggle to provide comprehensive control over the color schemes of an entire image, especially for background elements and less prominent objects not explicitly mentioned in prompts. This paper proposes a novel approach to enhance color scheme control by integrating color palettes as a separate guidance mechanism alongside prompt instructions. We investigate the effectiveness of palette guidance by exploring various palette representation methods within a diffusion-based image colorization framework. To facilitate this exploration, we construct specialized palette-text-image datasets and conduct extensive quantitative and qualitative analyses. Our results demonstrate that incorporating palette guidance significantly improves the model's ability to generate images with desired color schemes, enabling a more controlled and refined colorization process.