Enhanced Variational Inference with Dyadic Transformation
This work addresses a specific bottleneck in variational inference for researchers, but it is incremental as it builds on existing VAE enhancements.
The paper tackled the limited flexibility of variational autoencoders' diagonal covariance latent variables by proposing a dyadic transformation to model multivariate normal distributions, achieving competitive results on the MNIST dataset.
Variational autoencoder is a powerful deep generative model with variational inference. The practice of modeling latent variables in the VAE's original formulation as normal distributions with a diagonal covariance matrix limits the flexibility to match the true posterior distribution. We propose a new transformation, dyadic transformation (DT), that can model a multivariate normal distribution. DT is a single-stage transformation with low computational requirements. We demonstrate empirically on MNIST dataset that DT enhances the posterior flexibility and attains competitive results compared to other VAE enhancements.