Improving Heterogeneous Face Recognition with Conditional Adversarial Networks
This addresses a practical problem in real-world applications where shape information is limited, offering an incremental improvement over existing cross-modal approaches.
The paper tackles heterogeneous face recognition between color and depth images by proposing a cross-modal deep learning method that uses CNNs for feature extraction and cGANs for depth reconstruction, achieving state-of-the-art performance on the FRGC 2D/3D database with improved efficiency.
Heterogeneous face recognition between color image and depth image is a much desired capacity for real world applications where shape information is looked upon as merely involved in gallery. In this paper, we propose a cross-modal deep learning method as an effective and efficient workaround for this challenge. Specifically, we begin with learning two convolutional neural networks (CNNs) to extract 2D and 2.5D face features individually. Once trained, they can serve as pre-trained models for another two-way CNN which explores the correlated part between color and depth for heterogeneous matching. Compared with most conventional cross-modal approaches, our method additionally conducts accurate depth image reconstruction from single color image with Conditional Generative Adversarial Nets (cGAN), and further enhances the recognition performance by fusing multi-modal matching results. Through both qualitative and quantitative experiments on benchmark FRGC 2D/3D face database, we demonstrate that the proposed pipeline outperforms state-of-the-art performance on heterogeneous face recognition and ensures a drastically efficient on-line stage.