Elastic Interaction Energy-Based Generative Model: Approximation in Feature Space
This work addresses instability and mode collapse issues in GANs, which are critical for generating diverse and high-quality data in machine learning applications, though it appears incremental as it builds upon existing GAN frameworks.
The authors tackled the problem of mode collapse and unstable training in generative adversarial networks (GANs) by proposing a loss function based on elastic interaction energy (EIE) and approximating distributions in a latent feature space. Their EIEG GAN model demonstrated improved stability and performance on datasets like MNIST, FashionMNIST, CIFAR-10, and CelebA, effectively mitigating mode collapse.
In this paper, we propose a novel approach to generative modeling using a loss function based on elastic interaction energy (EIE), which is inspired by the elastic interaction between defects in crystals. The utilization of the EIE-based metric presents several advantages, including its long range property that enables consideration of global information in the distribution. Moreover, its inclusion of a self-interaction term helps to prevent mode collapse and captures all modes of distribution. To overcome the difficulty of the relatively scattered distribution of high-dimensional data, we first map the data into a latent feature space and approximate the feature distribution instead of the data distribution. We adopt the GAN framework and replace the discriminator with a feature transformation network to map the data into a latent space. We also add a stabilizing term to the loss of the feature transformation network, which effectively addresses the issue of unstable training in GAN-based algorithms. Experimental results on popular datasets, such as MNIST, FashionMNIST, CIFAR-10, and CelebA, demonstrate that our EIEG GAN model can mitigate mode collapse, enhance stability, and improve model performance.