Exemplar VAE: Linking Generative Models, Nearest Neighbor Retrieval, and Data Augmentation
This work addresses the challenge of combining generative models with nearest-neighbor retrieval for researchers in machine learning, offering incremental improvements in data augmentation and representation learning.
The paper tackles the problem of bridging parametric and non-parametric generative models by introducing Exemplar VAEs, which use a non-parametric prior and retrieval-augmented training, resulting in improved density estimation and representation learning, with generative data augmentation reducing classification error on MNIST from 1.17% to 0.69% and on Fashion MNIST from 8.56% to 8.16%.
We introduce Exemplar VAEs, a family of generative models that bridge the gap between parametric and non-parametric, exemplar based generative models. Exemplar VAE is a variant of VAE with a non-parametric prior in the latent space based on a Parzen window estimator. To sample from it, one first draws a random exemplar from a training set, then stochastically transforms that exemplar into a latent code and a new observation. We propose retrieval augmented training (RAT) as a way to speed up Exemplar VAE training by using approximate nearest neighbor search in the latent space to define a lower bound on log marginal likelihood. To enhance generalization, model parameters are learned using exemplar leave-one-out and subsampling. Experiments demonstrate the effectiveness of Exemplar VAEs on density estimation and representation learning. Importantly, generative data augmentation using Exemplar VAEs on permutation invariant MNIST and Fashion MNIST reduces classification error from 1.17% to 0.69% and from 8.56% to 8.16%.