Realizing GANs via a Tunable Loss Function
This work offers a theoretical framework for GANs that could benefit researchers in generative modeling, but it appears incremental as it builds on existing divergence concepts without demonstrating practical gains.
The authors tackled the problem of improving GAN training by introducing a tunable GAN framework called α-GAN, which interpolates between different GAN types and addresses issues like vanishing gradients and mode collapse, though no concrete numerical results are provided.
We introduce a tunable GAN, called $α$-GAN, parameterized by $α\in (0,\infty]$, which interpolates between various $f$-GANs and Integral Probability Metric based GANs (under constrained discriminator set). We construct $α$-GAN using a supervised loss function, namely, $α$-loss, which is a tunable loss function capturing several canonical losses. We show that $α$-GAN is intimately related to the Arimoto divergence, which was first proposed by Österriecher (1996), and later studied by Liese and Vajda (2006). We also study the convergence properties of $α$-GAN. We posit that the holistic understanding that $α$-GAN introduces will have practical benefits of addressing both the issues of vanishing gradients and mode collapse.