JIFF: Jointly-aligned Implicit Face Function for High Quality Single View Clothed Human Reconstruction
This addresses the issue of poor face detail in 3D human reconstructions for applications such as telepresence, representing an incremental improvement over existing implicit function methods.
The paper tackles the problem of single-view 3D human reconstruction by improving face detail recovery, which degrades user experience in applications like 3D telepresence, and proposes JIFF to achieve superior performance over state-of-the-art methods in evaluations.
This paper addresses the problem of single view 3D human reconstruction. Recent implicit function based methods have shown impressive results, but they fail to recover fine face details in their reconstructions. This largely degrades user experience in applications like 3D telepresence. In this paper, we focus on improving the quality of face in the reconstruction and propose a novel Jointly-aligned Implicit Face Function (JIFF) that combines the merits of the implicit function based approach and model based approach. We employ a 3D morphable face model as our shape prior and compute space-aligned 3D features that capture detailed face geometry information. Such space-aligned 3D features are combined with pixel-aligned 2D features to jointly predict an implicit face function for high quality face reconstruction. We further extend our pipeline and introduce a coarse-to-fine architecture to predict high quality texture for our detailed face model. Extensive evaluations have been carried out on public datasets and our proposed JIFF has demonstrates superior performance (both quantitatively and qualitatively) over existing state-of-the-arts.