Conditioning Diffusion Models via Attributes and Semantic Masks for Face Generation
This work addresses the need for fine-grained control in face generation for applications like digital art or synthetic data, but it is incremental as it builds on existing diffusion model conditioning methods.
The paper tackles the problem of generating diverse and controllable face images by proposing a multi-conditioning approach for diffusion models using attributes and semantic masks, achieving realistic and diverse samples on the CelebA-HQ dataset.
Deep generative models have shown impressive results in generating realistic images of faces. GANs managed to generate high-quality, high-fidelity images when conditioned on semantic masks, but they still lack the ability to diversify their output. Diffusion models partially solve this problem and are able to generate diverse samples given the same condition. In this paper, we propose a multi-conditioning approach for diffusion models via cross-attention exploiting both attributes and semantic masks to generate high-quality and controllable face images. We also studied the impact of applying perceptual-focused loss weighting into the latent space instead of the pixel space. Our method extends the previous approaches by introducing conditioning on more than one set of features, guaranteeing a more fine-grained control over the generated face images. We evaluate our approach on the CelebA-HQ dataset, and we show that it can generate realistic and diverse samples while allowing for fine-grained control over multiple attributes and semantic regions. Additionally, we perform an ablation study to evaluate the impact of different conditioning strategies on the quality and diversity of the generated images.