DeepWarp: Photorealistic Image Resynthesis for Gaze Manipulation
This addresses the need for realistic gaze manipulation in applications like virtual reality or photo editing, representing a domain-specific incremental improvement.
The paper tackles the problem of generating photorealistic images of faces with redirected gaze by treating it as conditional image generation, achieving strong results as shown by numerical comparisons and a user study.
In this work, we consider the task of generating highly-realistic images of a given face with a redirected gaze. We treat this problem as a specific instance of conditional image generation and suggest a new deep architecture that can handle this task very well as revealed by numerical comparison with prior art and a user study. Our deep architecture performs coarse-to-fine warping with an additional intensity correction of individual pixels. All these operations are performed in a feed-forward manner, and the parameters associated with different operations are learned jointly in the end-to-end fashion. After learning, the resulting neural network can synthesize images with manipulated gaze, while the redirection angle can be selected arbitrarily from a certain range and provided as an input to the network.