Algorithms that get old : the case of generative deep neural networks
This addresses a limitation in generative models for machine learning, offering an incremental improvement to enhance diversity and coverage in generated outputs.
The paper tackles the problem that generative deep neural networks like VAEs and GANs produce repetitive outputs, unlike human artists who evolve their style over time, and proposes a numerical paradigm to ensure generated objects do not repeat and evolve to cover the entire target distribution.
Generative deep neural networks used in machine learning, like the Variational Auto-Encoders (VAE), and Generative Adversarial Networks (GANs) produce new objects each time when asked to do so with the constraint that the new objects remain similar to some list of examples given as input. However, this behavior is unlike that of human artists that change their style as time goes by and seldom return to the style of the initial creations. We investigate a situation where VAEs are used to sample from a probability measure described by some empirical dataset. Based on recent works on Radon-Sobolev statistical distances, we propose a numerical paradigm, to be used in conjunction with a generative algorithm, that satisfies the two following requirements: the objects created do not repeat and evolve to fill the entire target probability distribution.