Learning Hierarchical Priors in VAEs
This work addresses a specific bottleneck in VAEs for researchers in generative modeling, though it appears incremental as it builds on the Taming VAEs framework.
The authors tackled the problem of over-regularization in variational autoencoders by learning a hierarchical prior, resulting in a latent representation that matches the data manifold's topology and exhibits smoothness and simple explanatory factors.
We propose to learn a hierarchical prior in the context of variational autoencoders to avoid the over-regularisation resulting from a standard normal prior distribution. To incentivise an informative latent representation of the data, we formulate the learning problem as a constrained optimisation problem by extending the Taming VAEs framework to two-level hierarchical models. We introduce a graph-based interpolation method, which shows that the topology of the learned latent representation corresponds to the topology of the data manifold---and present several examples, where desired properties of latent representation such as smoothness and simple explanatory factors are learned by the prior.