Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks
This addresses the computational cost issue for real-time applications in graphics and vision, though it is incremental as it builds on existing neural texture synthesis methods.
The paper tackles the efficiency problem in neural texture synthesis by proposing Markovian Generative Adversarial Networks (MGANs), which precompute a feed-forward network to generate textures without optimization at runtime, achieving a run-time performance of 0.25M pixel images at 25Hz and being at least 500 times faster than previous methods.
This paper proposes Markovian Generative Adversarial Networks (MGANs), a method for training generative neural networks for efficient texture synthesis. While deep neural network approaches have recently demonstrated remarkable results in terms of synthesis quality, they still come at considerable computational costs (minutes of run-time for low-res images). Our paper addresses this efficiency issue. Instead of a numerical deconvolution in previous work, we precompute a feed-forward, strided convolutional network that captures the feature statistics of Markovian patches and is able to directly generate outputs of arbitrary dimensions. Such network can directly decode brown noise to realistic texture, or photos to artistic paintings. With adversarial training, we obtain quality comparable to recent neural texture synthesis methods. As no optimization is required any longer at generation time, our run-time performance (0.25M pixel images at 25Hz) surpasses previous neural texture synthesizers by a significant margin (at least 500 times faster). We apply this idea to texture synthesis, style transfer, and video stylization.