Three Variations on Variational Autoencoders
This work is incremental, addressing convergence analysis for VAEs, which could benefit researchers in generative modeling.
The authors tackled the problem of improving variational autoencoders (VAEs) by developing three variations, including one that introduces an Evidence Upper Bound (EUBO) to assess convergence, but no concrete numerical results or performance gains are reported.
Variational autoencoders (VAEs) are one class of generative probabilistic latent-variable models designed for inference based on known data. We develop three variations on VAEs by introducing a second parameterized encoder/decoder pair and, for one variation, an additional fixed encoder. The parameters of the encoders/decoders are to be learned with a neural network. The fixed encoder is obtained by probabilistic-PCA. The variations are compared to the Evidence Lower Bound (ELBO) approximation to the original VAE. One variation leads to an Evidence Upper Bound (EUBO) that can be used in conjunction with the original ELBO to interrogate the convergence of the VAE.