High-Quality 3D Face Reconstruction with Affine Convolutional Networks
This work addresses a specific bottleneck in 3D face reconstruction for computer vision applications, offering an incremental improvement over existing methods.
The paper tackles the problem of spatial misalignment between input images and canonical outputs in 3D face reconstruction by proposing Affine Convolutional Networks, which learn affine transformations to handle non-corresponding inputs and outputs, achieving high-quality UV maps at 512 x 512 pixels compared to previous 256 x 256 or smaller resolutions.
Recent works based on convolutional encoder-decoder architecture and 3DMM parameterization have shown great potential for canonical view reconstruction from a single input image. Conventional CNN architectures benefit from exploiting the spatial correspondence between the input and output pixels. However, in 3D face reconstruction, the spatial misalignment between the input image (e.g. face) and the canonical/UV output makes the feature encoding-decoding process quite challenging. In this paper, to tackle this problem, we propose a new network architecture, namely the Affine Convolution Networks, which enables CNN based approaches to handle spatially non-corresponding input and output images and maintain high-fidelity quality output at the same time. In our method, an affine transformation matrix is learned from the affine convolution layer for each spatial location of the feature maps. In addition, we represent 3D human heads in UV space with multiple components, including diffuse maps for texture representation, position maps for geometry representation, and light maps for recovering more complex lighting conditions in the real world. All the components can be trained without any manual annotations. Our method is parametric-free and can generate high-quality UV maps at resolution of 512 x 512 pixels, while previous approaches normally generate 256 x 256 pixels or smaller. Our code will be released once the paper got accepted.