Smart, Sparse Contours to Represent and Edit Images
This work addresses the challenge of efficient and intuitive image representation and editing for applications in computer vision and graphics, offering a novel approach but with incremental improvements in sparsity and manipulation capabilities.
The paper tackles the problem of reconstructing images from sparse contour data, achieving high-quality reconstructions with less than 6% of image pixels, which improves over existing methods requiring denser input. It enables semantically-aware image manipulation, such as changing facial expressions through simple contour edits, with reconstructions often indistinguishable from source images by humans and face recognition systems.
We study the problem of reconstructing an image from information stored at contour locations. We show that high-quality reconstructions with high fidelity to the source image can be obtained from sparse input, e.g., comprising less than $6\%$ of image pixels. This is a significant improvement over existing contour-based reconstruction methods that require much denser input to capture subtle texture information and to ensure image quality. Our model, based on generative adversarial networks, synthesizes texture and details in regions where no input information is provided. The semantic knowledge encoded into our model and the sparsity of the input allows to use contours as an intuitive interface for semantically-aware image manipulation: local edits in contour domain translate to long-range and coherent changes in pixel space. We can perform complex structural changes such as changing facial expression by simple edits of contours. Our experiments demonstrate that humans as well as a face recognition system mostly cannot distinguish between our reconstructions and the source images.