Symmetric Equilibrium Learning of VAEs
This addresses a problem for researchers and practitioners in machine learning by enabling more flexible VAE applications in complex scenarios like semi-supervised learning, though it appears incremental as it builds on existing VAE frameworks.
The paper tackles the asymmetry and limitations of standard variational autoencoder (VAE) learning, which relies on maximizing the evidence lower bound (ELBO) and requires a closed-form prior, by proposing a symmetric Nash equilibrium learning approach that enables learning in scenarios where both data and latent distributions are accessible only by sampling.
We view variational autoencoders (VAE) as decoder-encoder pairs, which map distributions in the data space to distributions in the latent space and vice versa. The standard learning approach for VAEs is the maximisation of the evidence lower bound (ELBO). It is asymmetric in that it aims at learning a latent variable model while using the encoder as an auxiliary means only. Moreover, it requires a closed form a-priori latent distribution. This limits its applicability in more complex scenarios, such as general semi-supervised learning and employing complex generative models as priors. We propose a Nash equilibrium learning approach, which is symmetric with respect to the encoder and decoder and allows learning VAEs in situations where both the data and the latent distributions are accessible only by sampling. The flexibility and simplicity of this approach allows its application to a wide range of learning scenarios and downstream tasks.