DCFace: Synthetic Face Generation with Dual Condition Diffusion Model
This work addresses the problem of dataset scarcity for face recognition researchers by providing a method to generate controlled synthetic data, though it is incremental as it builds on diffusion models for a specific domain.
The paper tackles the challenge of generating synthetic face datasets for training face recognition models by proposing DCFace, a dual condition diffusion model that controls subject appearance and external factors like pose and illumination, resulting in a 6.11% average improvement in verification accuracy across four out of five test datasets.
Generating synthetic datasets for training face recognition models is challenging because dataset generation entails more than creating high fidelity images. It involves generating multiple images of same subjects under different factors (\textit{e.g.}, variations in pose, illumination, expression, aging and occlusion) which follows the real image conditional distribution. Previous works have studied the generation of synthetic datasets using GAN or 3D models. In this work, we approach the problem from the aspect of combining subject appearance (ID) and external factor (style) conditions. These two conditions provide a direct way to control the inter-class and intra-class variations. To this end, we propose a Dual Condition Face Generator (DCFace) based on a diffusion model. Our novel Patch-wise style extractor and Time-step dependent ID loss enables DCFace to consistently produce face images of the same subject under different styles with precise control. Face recognition models trained on synthetic images from the proposed DCFace provide higher verification accuracies compared to previous works by $6.11\%$ on average in $4$ out of $5$ test datasets, LFW, CFP-FP, CPLFW, AgeDB and CALFW. Code is available at https://github.com/mk-minchul/dcface