Deep-MDS Framework for Recovering the 3D Shape of 2D Landmarks from a Single Image
This work addresses 3D face reconstruction from single images, which is important for applications in computer vision and graphics, but it is incremental as it builds on existing NMDS and deep learning techniques.
The paper tackles the problem of recovering 3D facial shapes from 2D landmarks in a single image by proposing a low-parameter deep learning framework that uses Non-metric Multi-Dimensional Scaling (NMDS) for the first time in this context, achieving comparable performance to state-of-the-art methods in efficiency and accuracy on datasets like BFM and CelebA.
In this paper, a low parameter deep learning framework utilizing the Non-metric Multi-Dimensional scaling (NMDS) method, is proposed to recover the 3D shape of 2D landmarks on a human face, in a single input image. Hence, NMDS approach is used for the first time to establish a mapping from a 2D landmark space to the corresponding 3D shape space. A deep neural network learns the pairwise dissimilarity among 2D landmarks, used by NMDS approach, whose objective is to learn the pairwise 3D Euclidean distance of the corresponding 2D landmarks on the input image. This scheme results in a symmetric dissimilarity matrix, with the rank larger than 2, leading the NMDS approach toward appropriately recovering the 3D shape of corresponding 2D landmarks. In the case of posed images and complex image formation processes like perspective projection which causes occlusion in the input image, we consider an autoencoder component in the proposed framework, as an occlusion removal part, which turns different input views of the human face into a profile view. The results of a performance evaluation using different synthetic and real-world human face datasets, including Besel Face Model (BFM), CelebA, CoMA - FLAME, and CASIA-3D, indicates the comparable performance of the proposed framework, despite its small number of training parameters, with the related state-of-the-art and powerful 3D reconstruction methods from the literature, in terms of efficiency and accuracy.