ED-VAE: Entropy Decomposition of ELBO in Variational Autoencoders
This addresses a methodological bottleneck in VAEs for researchers and practitioners working with complex priors, though it appears incremental as it reformulates rather than replaces the core VAE framework.
The paper tackles the limitations of Variational Autoencoders (VAEs) due to constraints in the Evidence Lower Bound (ELBO) formulation, particularly with complex priors, by introducing ED-VAE, a novel ELBO reformulation that explicitly includes entropy and cross-entropy components, which enhances model flexibility and improves generative performance.
Traditional Variational Autoencoders (VAEs) are constrained by the limitations of the Evidence Lower Bound (ELBO) formulation, particularly when utilizing simplistic, non-analytic, or unknown prior distributions. These limitations inhibit the VAE's ability to generate high-quality samples and provide clear, interpretable latent representations. This work introduces the Entropy Decomposed Variational Autoencoder (ED-VAE), a novel re-formulation of the ELBO that explicitly includes entropy and cross-entropy components. This reformulation significantly enhances model flexibility, allowing for the integration of complex and non-standard priors. By providing more detailed control over the encoding and regularization of latent spaces, ED-VAE not only improves interpretability but also effectively captures the complex interactions between latent variables and observed data, thus leading to better generative performance.