Self-Creative Text-to-Object Generation using Semantic-Aware Spatial Weighting
For researchers and practitioners in text-to-image generation, this work addresses the challenge of producing creative and novel images beyond literal text-image alignment.
The paper tackles the lack of creativity in text-to-image generation by proposing a Self-Creative Diffusion (SCDiff) model with a learnable spatial weighting module and a visual-semantic mixing loss, achieving substantial improvements in creativity, semantic alignment, and visual coherence.
Instilling creativity in text-to-image (T2I) generation presents a significant challenge, as it requires synthesized images to exhibit not only visual novelty and surprise, but also artistic value. Current T2I models, however, are largely optimized for literal text-image alignment with their data distribution, and their noise prediction networks constrain the generation to high-probability regions, consequently generating outputs that lack authentic creativity. To address this, we propose a Self-Creative Diffusion (SCDiff) model for meaningful T2I generations featuring two core modules: a learnable spatial weighting (LSW) module and a visual-semantic mixing loss (VSML). The LSW module designs a parametric Kaiser-Bessel window to reinforce central image features, fostering novel and surprising generation. The VSML module introduces a dual loss function: a similarity loss constrains that the new images align with its textual description, while a diversity loss maximizes its distinction from the original image, enhancing both semantic value and visual novelty. Extensive experiments demonstrate that our model substantially improves creativity, semantic alignment, and visual coherence, offering a simple yet powerful framework for generating creative objects.