PI-REC: Progressive Image Reconstruction Network With Edge and Color Domain
This work addresses image reconstruction for applications like translating hand-drawn drafts into paintings, but it is incremental as it builds on existing GAN-based methods.
The authors tackled the problem of reconstructing detailed images from sparse binary edges and flat color inputs, achieving state-of-the-art results in edge-to-image translation with improved realism and accuracy.
We propose a universal image reconstruction method to represent detailed images purely from binary sparse edge and flat color domain. Inspired by the procedures of painting, our framework, based on generative adversarial network, consists of three phases: Imitation Phase aims at initializing networks, followed by Generating Phase to reconstruct preliminary images. Moreover, Refinement Phase is utilized to fine-tune preliminary images into final outputs with details. This framework allows our model generating abundant high frequency details from sparse input information. We also explore the defects of disentangling style latent space implicitly from images, and demonstrate that explicit color domain in our model performs better on controllability and interpretability. In our experiments, we achieve outstanding results on reconstructing realistic images and translating hand drawn drafts into satisfactory paintings. Besides, within the domain of edge-to-image translation, our model PI-REC outperforms existing state-of-the-art methods on evaluations of realism and accuracy, both quantitatively and qualitatively.