Semi-parametric Image Synthesis
This addresses the challenge of high-quality image synthesis for applications like computer vision and graphics, though it appears incremental as it builds on existing parametric and nonparametric approaches.
The paper tackles the problem of generating realistic photographic images from semantic layouts by combining parametric and nonparametric techniques, resulting in considerably more realistic images than recent purely parametric methods on multiple datasets.
We present a semi-parametric approach to photographic image synthesis from semantic layouts. The approach combines the complementary strengths of parametric and nonparametric techniques. The nonparametric component is a memory bank of image segments constructed from a training set of images. Given a novel semantic layout at test time, the memory bank is used to retrieve photographic references that are provided as source material to a deep network. The synthesis is performed by a deep network that draws on the provided photographic material. Experiments on multiple semantic segmentation datasets show that the presented approach yields considerably more realistic images than recent purely parametric techniques. The results are shown in the supplementary video at https://youtu.be/U4Q98lenGLQ