Exploring the structure of a real-time, arbitrary neural artistic stylization network
This enables real-time arbitrary artistic stylization for applications like video processing and interactive tools, though it builds incrementally on existing multi-style transfer methods.
The paper tackles the problem of real-time artistic style transfer for arbitrary content/style image pairs by combining neural style algorithm flexibility with fast style transfer speed, achieving successful training on 80,000 paintings and demonstrating generalization to unseen paintings with a smooth, semantically organized embedding space.
In this paper, we present a method which combines the flexibility of the neural algorithm of artistic style with the speed of fast style transfer networks to allow real-time stylization using any content/style image pair. We build upon recent work leveraging conditional instance normalization for multi-style transfer networks by learning to predict the conditional instance normalization parameters directly from a style image. The model is successfully trained on a corpus of roughly 80,000 paintings and is able to generalize to paintings previously unobserved. We demonstrate that the learned embedding space is smooth and contains a rich structure and organizes semantic information associated with paintings in an entirely unsupervised manner.