BioFaceNet: Deep Biophysical Face Image Interpretation
This addresses the challenge of biophysical face interpretation for computer vision applications, but it is incremental as it builds on existing model-based priors and self-supervised methods.
The paper tackles the problem of decomposing a single face image into biophysical parameters, shading maps, and camera/illumination properties using a deep CNN called BioFaceNet, achieving convincing qualitative results on in-the-wild data and introducing a benchmark for quantitative evaluation.
In this paper we present BioFaceNet, a deep CNN that learns to decompose a single face image into biophysical parameters maps, diffuse and specular shading maps as well as estimating the spectral power distribution of the scene illuminant and the spectral sensitivity of the camera. The network comprises a fully convolutional encoder for estimating the spatial maps with a fully connected branch for estimating the vector quantities. The network is trained using a self-supervised appearance loss computed via a model-based decoder. The task is highly underconstrained so we impose a number of model-based priors. Skin spectral reflectance is restricted to a biophysical model, we impose a statistical prior on camera spectral sensitivities, a physical constraint on illumination spectra, a sparsity prior on specular reflections and direct supervision on diffuse shading using a rough shape proxy. We show convincing qualitative results on in-the-wild data and introduce a benchmark for quantitative evaluation on this new task.