CLIP-CLOP: CLIP-Guided Collage and Photomontage
This work addresses artists seeking greater creative freedom in AI-assisted art generation, though it is incremental in building upon existing CLIP-based methods.
The authors tackled the problem of limited human control in AI-generated art by developing a gradient-based generator for creating collages, which allows artists to curate image patches and adjust positions during generation, resulting in an open-source tool for high-resolution collage creation.
The unabated mystique of large-scale neural networks, such as the CLIP dual image-and-text encoder, popularized automatically generated art. Increasingly more sophisticated generators enhanced the artworks' realism and visual appearance, and creative prompt engineering enabled stylistic expression. Guided by an artist-in-the-loop ideal, we design a gradient-based generator to produce collages. It requires the human artist to curate libraries of image patches and to describe (with prompts) the whole image composition, with the option to manually adjust the patches' positions during generation, thereby allowing humans to reclaim some control of the process and achieve greater creative freedom. We explore the aesthetic potentials of high-resolution collages, and provide an open-source Google Colab as an artistic tool.