SUPER: Selfie Undistortion and Head Pose Editing with Identity Preservation
This addresses the issue of unattractive selfies for users, but it is incremental as it builds on existing 3D GAN inversion and blending techniques.
The paper tackles the problem of unnatural selfies caused by close-up distortions and poor head poses by proposing SUPER, a method that eliminates distortions and adjusts head pose while preserving identity, achieving superior performance on benchmarks and a new dataset.
Self-portraits captured from a short distance might look unnatural or even unattractive due to heavy distortions making facial features malformed, and ill-placed head poses. In this paper, we propose SUPER, a novel method of eliminating distortions and adjusting head pose in a close-up face crop. We perform 3D GAN inversion for a facial image by optimizing camera parameters and face latent code, which gives a generated image. Besides, we estimate depth from the obtained latent code, create a depth-induced 3D mesh, and render it with updated camera parameters to obtain a warped portrait. Finally, we apply the visibility-based blending so that visible regions are reprojected, and occluded parts are restored with a generative model. Experiments on face undistortion benchmarks and on our self-collected Head Rotation dataset (HeRo), show that SUPER outperforms previous approaches both qualitatively and quantitatively, opening new possibilities for photorealistic selfie editing.