On some theoretical limitations of Generative Adversarial Networks
This addresses a foundational theoretical gap for researchers and practitioners in machine learning, revealing a critical limitation in GANs' capabilities.
The paper tackles the problem of theoretical limitations in Generative Adversarial Networks (GANs), specifically showing that GANs cannot generate heavy-tailed distributions, as proven using Extreme Value Theory.
Generative Adversarial Networks have become a core technique in Machine Learning to generate unknown distributions from data samples. They have been used in a wide range of context without paying much attention to the possible theoretical limitations of those models. Indeed, because of the universal approximation properties of Neural Networks, it is a general assumption that GANs can generate any probability distribution. Recently, people began to question this assumption and this article is in line with this thinking. We provide a new result based on Extreme Value Theory showing that GANs can't generate heavy tailed distributions. The full proof of this result is given.