Redefining <Creative> in Dictionary: Towards an Enhanced Semantic Understanding of Creative Generation
This addresses a specific bottleneck in creative generation for AI art and design applications, offering a more efficient and universal solution compared to existing retraining-based methods.
The paper tackles the problem of text-to-image diffusion models struggling to generate combinatorial creative concepts like 'a creative mixture that resembles a lettuce and a mantis' by introducing CreTok, a method that redefines 'creative' as a new token to enhance semantic understanding, achieving state-of-the-art performance with improved text-image alignment and higher human preference ratings without additional training.
``Creative'' remains an inherently abstract concept for both humans and diffusion models. While text-to-image (T2I) diffusion models can easily generate out-of-distribution concepts like ``a blue banana'', they struggle with generating combinatorial objects such as ``a creative mixture that resembles a lettuce and a mantis'', due to difficulties in understanding the semantic depth of ``creative''. Current methods rely heavily on synthesizing reference prompts or images to achieve a creative effect, typically requiring retraining for each unique creative output-a process that is computationally intensive and limits practical applications. To address this, we introduce CreTok, which brings meta-creativity to diffusion models by redefining ``creative'' as a new token, \texttt{<CreTok>}, thus enhancing models' semantic understanding for combinatorial creativity. CreTok achieves such redefinition by iteratively sampling diverse text pairs from our proposed CangJie dataset to form adaptive prompts and restrictive prompts, and then optimizing the similarity between their respective text embeddings. Extensive experiments demonstrate that <CreTok> enables the universal and direct generation of combinatorial creativity across diverse concepts without additional training, achieving state-of-the-art performance with improved text-image alignment and higher human preference ratings. Code will be made available at https://github.com/fu-feng/CreTok.