Batch norm with entropic regularization turns deterministic autoencoders into generative models
This provides a simpler alternative to variational autoencoders for generative modeling, potentially benefiting researchers and practitioners in machine learning.
The paper tackles the problem of turning deterministic autoencoders into generative models by using batch normalization as a source of non-determinism and adding entropic regularization, achieving performance on par with variational autoencoders.
The variational autoencoder is a well defined deep generative model that utilizes an encoder-decoder framework where an encoding neural network outputs a non-deterministic code for reconstructing an input. The encoder achieves this by sampling from a distribution for every input, instead of outputting a deterministic code per input. The great advantage of this process is that it allows the use of the network as a generative model for sampling from the data distribution beyond provided samples for training. We show in this work that utilizing batch normalization as a source for non-determinism suffices to turn deterministic autoencoders into generative models on par with variational ones, so long as we add a suitable entropic regularization to the training objective.