TCDiff: Triple Condition Diffusion Model with 3D Constraints for Stylizing Synthetic Faces
This addresses the need for ethical and privacy-compliant synthetic face data to train robust face recognition models, offering an incremental improvement over existing methods.
The paper tackles the problem of generating synthetic face datasets with high identity consistency and intra-class variance for face recognition training by proposing TCDiff, a triple condition diffusion model with 2D and 3D constraints, which outperforms state-of-the-art synthetic datasets on real benchmarks like LFW, CFP-FP, AgeDB, and BUPT.
A robust face recognition model must be trained using datasets that include a large number of subjects and numerous samples per subject under varying conditions (such as pose, expression, age, noise, and occlusion). Due to ethical and privacy concerns, large-scale real face datasets have been discontinued, such as MS1MV3, and synthetic face generators have been proposed, utilizing GANs and Diffusion Models, such as SYNFace, SFace, DigiFace-1M, IDiff-Face, DCFace, and GANDiffFace, aiming to supply this demand. Some of these methods can produce high-fidelity realistic faces, but with low intra-class variance, while others generate high-variance faces with low identity consistency. In this paper, we propose a Triple Condition Diffusion Model (TCDiff) to improve face style transfer from real to synthetic faces through 2D and 3D facial constraints, enhancing face identity consistency while keeping the necessary high intra-class variance. Face recognition experiments using 1k, 2k, and 5k classes of our new dataset for training outperform state-of-the-art synthetic datasets in real face benchmarks such as LFW, CFP-FP, AgeDB, and BUPT. Our source code is available at: https://github.com/BOVIFOCR/tcdiff.