Noise Dimension of GAN: An Image Compression Perspective
This work addresses a theoretical gap in GAN design for researchers, but it is incremental as it builds on existing GAN frameworks without broad practical application changes.
The paper tackles the problem of understanding the required noise dimension in GANs by proposing a view of GANs as discrete samplers, linking it to image compression bits, and introduces a divergence-entropy trade-off to analyze limited noise, with experimental verification on image generation.
Generative adversial network (GAN) is a type of generative model that maps a high-dimensional noise to samples in target distribution. However, the dimension of noise required in GAN is not well understood. Previous approaches view GAN as a mapping from a continuous distribution to another continous distribution. In this paper, we propose to view GAN as a discrete sampler instead. From this perspective, we build a connection between the minimum noise required and the bits to losslessly compress the images. Furthermore, to understand the behaviour of GAN when noise dimension is limited, we propose divergence-entropy trade-off. This trade-off depicts the best divergence we can achieve when noise is limited. And as rate distortion trade-off, it can be numerically solved when source distribution is known. Finally, we verifies our theory with experiments on image generation.