Conditional Distribution Modelling for Few-Shot Image Synthesis with Diffusion Models
This work addresses a bottleneck in few-shot image generation for computer vision applications, offering an incremental improvement over existing methods.
The paper tackles the problem of limited diversity in few-shot image synthesis by proposing Conditional Distribution Modelling (CDM), which leverages diffusion models and training data statistics to better approximate unseen class distributions, resulting in improved generation performance on four benchmark datasets.
Few-shot image synthesis entails generating diverse and realistic images of novel categories using only a few example images. While multiple recent efforts in this direction have achieved impressive results, the existing approaches are dependent only upon the few novel samples available at test time in order to generate new images, which restricts the diversity of the generated images. To overcome this limitation, we propose Conditional Distribution Modelling (CDM) -- a framework which effectively utilizes Diffusion models for few-shot image generation. By modelling the distribution of the latent space used to condition a Diffusion process, CDM leverages the learnt statistics of the training data to get a better approximation of the unseen class distribution, thereby removing the bias arising due to limited number of few shot samples. Simultaneously, we devise a novel inversion based optimization strategy that further improves the approximated unseen class distribution, and ensures the fidelity of the generated samples to the unseen class. The experimental results on four benchmark datasets demonstrate the effectiveness of our proposed CDM for few-shot generation.