Joint Bilateral Learning for Real-time Universal Photorealistic Style Transfer
This addresses the need for fast, high-quality photorealistic style transfer for applications like mobile photography, though it is incremental as it builds on existing deep learning approaches.
The paper tackles the problem of slow speed and artifacts in photorealistic style transfer by proposing a feed-forward neural network that learns local edge-aware affine transforms, achieving results three orders of magnitude faster than state-of-the-art methods and enabling real-time performance at 4K on a mobile phone.
Photorealistic style transfer is the task of transferring the artistic style of an image onto a content target, producing a result that is plausibly taken with a camera. Recent approaches, based on deep neural networks, produce impressive results but are either too slow to run at practical resolutions, or still contain objectionable artifacts. We propose a new end-to-end model for photorealistic style transfer that is both fast and inherently generates photorealistic results. The core of our approach is a feed-forward neural network that learns local edge-aware affine transforms that automatically obey the photorealism constraint. When trained on a diverse set of images and a variety of styles, our model can robustly apply style transfer to an arbitrary pair of input images. Compared to the state of the art, our method produces visually superior results and is three orders of magnitude faster, enabling real-time performance at 4K on a mobile phone. We validate our method with ablation and user studies.