Implicit Manifold Learning on Generative Adversarial Networks
This provides theoretical insights into GAN training dynamics, addressing mode collapse and distribution matching for researchers in generative modeling, though it is incremental as it builds on existing divergence analyses.
The paper tackles the problem of understanding how GANs learn data distributions by analyzing the support matching between learned and real data manifolds under different divergence metrics, showing that Jensen-Shannon divergence forces perfect matching while Wasserstein distance does not, and conjecturing that Wasserstein W2^2 may reduce mode collapse.
This paper raises an implicit manifold learning perspective in Generative Adversarial Networks (GANs), by studying how the support of the learned distribution, modelled as a submanifold $\mathcal{M}_θ$, perfectly match with $\mathcal{M}_{r}$, the support of the real data distribution. We show that optimizing Jensen-Shannon divergence forces $\mathcal{M}_θ$ to perfectly match with $\mathcal{M}_{r}$, while optimizing Wasserstein distance does not. On the other hand, by comparing the gradients of the Jensen-Shannon divergence and the Wasserstein distances ($W_1$ and $W_2^2$) in their primal forms, we conjecture that Wasserstein $W_2^2$ may enjoy desirable properties such as reduced mode collapse. It is therefore interesting to design new distances that inherit the best from both distances.