Semantic Style Transfer and Turning Two-Bit Doodles into Fine Artworks
This work addresses the problem of making generative AI tools more controllable and intuitive for users, particularly in creative domains like art and design, though it appears incremental as it builds on existing CNN methods with semantic augmentation.
The paper tackles the challenge of unpredictable behavior in convolutional neural networks for image synthesis and style transfer by introducing semantic annotations to create a content-aware generative algorithm, resulting in increased image quality, more plausible outcomes, and extended functional range for applications like semantic style transfer and transforming doodles into artworks.
Convolutional neural networks (CNNs) have proven highly effective at image synthesis and style transfer. For most users, however, using them as tools can be a challenging task due to their unpredictable behavior that goes against common intuitions. This paper introduces a novel concept to augment such generative architectures with semantic annotations, either by manually authoring pixel labels or using existing solutions for semantic segmentation. The result is a content-aware generative algorithm that offers meaningful control over the outcome. Thus, we increase the quality of images generated by avoiding common glitches, make the results look significantly more plausible, and extend the functional range of these algorithms---whether for portraits or landscapes, etc. Applications include semantic style transfer and turning doodles with few colors into masterful paintings!