FewGAN: Generating from the Joint Distribution of a Few Images
This addresses the challenge of limited data for image generation, though it appears incremental as an adaptation of existing patch-GAN and VQ-GAN methods.
The authors tackled the problem of generating novel, high-quality images from only a few training samples by developing FewGAN, a hierarchical patch-GAN that uses quantization and autoregressive modeling, resulting in superior performance over baselines in quantitative and qualitative experiments.
We introduce FewGAN, a generative model for generating novel, high-quality and diverse images whose patch distribution lies in the joint patch distribution of a small number of N>1 training samples. The method is, in essence, a hierarchical patch-GAN that applies quantization at the first coarse scale, in a similar fashion to VQ-GAN, followed by a pyramid of residual fully convolutional GANs at finer scales. Our key idea is to first use quantization to learn a fixed set of patch embeddings for training images. We then use a separate set of side images to model the structure of generated images using an autoregressive model trained on the learned patch embeddings of training images. Using quantization at the coarsest scale allows the model to generate both conditional and unconditional novel images. Subsequently, a patch-GAN renders the fine details, resulting in high-quality images. In an extensive set of experiments, it is shown that FewGAN outperforms baselines both quantitatively and qualitatively.