A Non-Adversarial Approach to Idempotent Generative Modelling
This addresses training issues in generative modeling for researchers, but it is incremental as it builds on existing methods like IGNs and IMLE.
The paper tackled mode collapse and instability in Idempotent Generative Networks by introducing a non-adversarial approach using Implicit Maximum Likelihood Estimation, resulting in improved data restoration and sample generation that closely matches the data distribution.
Idempotent Generative Networks (IGNs) are deep generative models that also function as local data manifold projectors, mapping arbitrary inputs back onto the manifold. They are trained to act as identity operators on the data and as idempotent operators off the data manifold. However, IGNs suffer from mode collapse, mode dropping, and training instability due to their objectives, which contain adversarial components and can cause the model to cover the data manifold only partially -- an issue shared with generative adversarial networks. We introduce Non-Adversarial Idempotent Generative Networks (NAIGNs) to address these issues. Our loss function combines reconstruction with the non-adversarial generative objective of Implicit Maximum Likelihood Estimation (IMLE). This improves on IGN's ability to restore corrupted data and generate new samples that closely match the data distribution. We moreover demonstrate that NAIGNs implicitly learn the distance field to the data manifold, as well as an energy-based model.