StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Synthesis
This addresses the problem of limited artistic flexibility in text-to-image generation for users in creative domains, representing an incremental improvement over existing methods.
The paper tackles the lack of artistic style control in text-to-drawing synthesis by introducing StyleCLIPDraw, which adds a style loss to CLIPDraw, enabling control over both content and style, with results showing that the coupled approach captures style in texture and shape.
Generating images that fit a given text description using machine learning has improved greatly with the release of technologies such as the CLIP image-text encoder model; however, current methods lack artistic control of the style of image to be generated. We introduce StyleCLIPDraw which adds a style loss to the CLIPDraw text-to-drawing synthesis model to allow artistic control of the synthesized drawings in addition to control of the content via text. Whereas performing decoupled style transfer on a generated image only affects the texture, our proposed coupled approach is able to capture a style in both texture and shape, suggesting that the style of the drawing is coupled with the drawing process itself. More results and our code are available at https://github.com/pschaldenbrand/StyleCLIPDraw