Variational Autoencoders Without the Variation
This work addresses generative modeling for researchers, but it is incremental as it builds on existing autoencoder approaches without introducing new methods.
The paper tackles the problem of generative modeling by exploring deterministic autoencoders (DAEs) for image generation without novel regularization methods, finding that DAEs can achieve successful results on CIFAR-10 and CelebA datasets with implicit regularization from large networks.
Variational autoencdoers (VAE) are a popular approach to generative modelling. However, exploiting the capabilities of VAEs in practice can be difficult. Recent work on regularised and entropic autoencoders have begun to explore the potential, for generative modelling, of removing the variational approach and returning to the classic deterministic autoencoder (DAE) with additional novel regularisation methods. In this paper we empirically explore the capability of DAEs for image generation without additional novel methods and the effect of the implicit regularisation and smoothness of large networks. We find that DAEs can be used successfully for image generation without additional loss terms, and that many of the useful properties of VAEs can arise implicitly from sufficiently large convolutional encoders and decoders when trained on CIFAR-10 and CelebA.