Diversity in deep generative models and generative AI
This addresses the issue of repetitive outputs in generative AI for researchers and practitioners, though it is incremental as it builds on existing sampling techniques.
The paper tackles the problem of limited diversity in deep generative models by introducing a kernel-based measure quantization method that approximates the target distribution as a whole and avoids repeating similar objects, resulting in improved diversity on classic benchmarks.
The decoder-based machine learning generative algorithms such as Generative Adversarial Networks (GAN), Variational Auto-Encoders (VAE), Transformers show impressive results when constructing objects similar to those in a training ensemble. However, the generation of new objects builds mainly on the understanding of the hidden structure of the training dataset followed by a sampling from a multi-dimensional normal variable. In particular each sample is independent from the others and can repeatedly propose same type of objects. To cure this drawback we introduce a kernel-based measure quantization method that can produce new objects from a given target measure by approximating it as a whole and even staying away from elements already drawn from that distribution. This ensures a better diversity of the produced objects. The method is tested on classic machine learning benchmarks.