Learning Perspective Undistortion of Portraits
This addresses a problem for computer vision and facial analysis applications by enabling more accurate processing of distorted portraits, though it is incremental as it builds on prior work in distortion correction.
The paper tackles the problem of perspective distortion artifacts in near-range portrait photographs, which bias perception and challenge facial tasks, by presenting a deep learning approach that predicts a distortion correction flow map to remove artifacts and infer missing features, significantly outperforming previous methods, particularly for extreme distortion or expressions, and improving face recognition accuracy and 3D reconstruction.
Near-range portrait photographs often contain perspective distortion artifacts that bias human perception and challenge both facial recognition and reconstruction techniques. We present the first deep learning based approach to remove such artifacts from unconstrained portraits. In contrast to the previous state-of-the-art approach, our method handles even portraits with extreme perspective distortion, as we avoid the inaccurate and error-prone step of first fitting a 3D face model. Instead, we predict a distortion correction flow map that encodes a per-pixel displacement that removes distortion artifacts when applied to the input image. Our method also automatically infers missing facial features, i.e. occluded ears caused by strong perspective distortion, with coherent details. We demonstrate that our approach significantly outperforms the previous state-of-the-art both qualitatively and quantitatively, particularly for portraits with extreme perspective distortion or facial expressions. We further show that our technique benefits a number of fundamental tasks, significantly improving the accuracy of both face recognition and 3D reconstruction and enables a novel camera calibration technique from a single portrait. Moreover, we also build the first perspective portrait database with a large diversity in identities, expression and poses, which will benefit the related research in this area.