Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis
This work addresses image synthesis challenges for computer vision and graphics applications, offering an incremental improvement by integrating existing techniques to enhance output quality.
The paper tackles the problem of synthesizing 2D images by combining generative Markov random fields (MRFs) with deep convolutional neural networks (dCNNs) to improve visual plausibility and reduce artifacts. The result is a method that outperforms classic generative MRF approaches, enabling synthesis of photographic and non-photo-realistic content with increased variability and adaptability.
This paper studies a combination of generative Markov random field (MRF) models and discriminatively trained deep convolutional neural networks (dCNNs) for synthesizing 2D images. The generative MRF acts on higher-levels of a dCNN feature pyramid, controling the image layout at an abstract level. We apply the method to both photographic and non-photo-realistic (artwork) synthesis tasks. The MRF regularizer prevents over-excitation artifacts and reduces implausible feature mixtures common to previous dCNN inversion approaches, permitting synthezing photographic content with increased visual plausibility. Unlike standard MRF-based texture synthesis, the combined system can both match and adapt local features with considerable variability, yielding results far out of reach of classic generative MRF methods.