Sliced-Wasserstein Autoencoder: An Embarrassingly Simple Generative Model
This provides a simpler alternative for generative modeling in machine learning, though it is incremental as it builds on existing autoencoder frameworks.
The paper tackles the problem of shaping the latent space distribution in generative autoencoders without adversarial training or closed-form distributions, by introducing Sliced-Wasserstein Autoencoders (SWAE) that use sliced-Wasserstein distance for regularization, resulting in a model with efficient numerical solution and simple implementation comparable to WAE and VAE.
In this paper we study generative modeling via autoencoders while using the elegant geometric properties of the optimal transport (OT) problem and the Wasserstein distances. We introduce Sliced-Wasserstein Autoencoders (SWAE), which are generative models that enable one to shape the distribution of the latent space into any samplable probability distribution without the need for training an adversarial network or defining a closed-form for the distribution. In short, we regularize the autoencoder loss with the sliced-Wasserstein distance between the distribution of the encoded training samples and a predefined samplable distribution. We show that the proposed formulation has an efficient numerical solution that provides similar capabilities to Wasserstein Autoencoders (WAE) and Variational Autoencoders (VAE), while benefiting from an embarrassingly simple implementation.