Cramer-Wold AutoEncoder
This is an incremental improvement for researchers and practitioners in generative modeling, as it simplifies training while maintaining quality.
The paper tackles the problem of simplifying generative models by introducing the Cramer-Wold AutoEncoder (CWAE), which uses a new Cramer-Wold metric to replace the Wasserstein metric, resulting in a simpler optimization procedure that produces samples with matching perceptual quality to state-of-the-art models.
We propose a new generative model, Cramer-Wold Autoencoder (CWAE). Following WAE, we directly encourage normality of the latent space. Our paper uses also the recent idea from Sliced WAE (SWAE) model, which uses one-dimensional projections as a method of verifying closeness of two distributions. The crucial new ingredient is the introduction of a new (Cramer-Wold) metric in the space of densities, which replaces the Wasserstein metric used in SWAE. We show that the Cramer-Wold metric between Gaussian mixtures is given by a simple analytic formula, which results in the removal of sampling necessary to estimate the cost function in WAE and SWAE models. As a consequence, while drastically simplifying the optimization procedure, CWAE produces samples of a matching perceptual quality to other SOTA models.