Funnels: Exact maximum likelihood with dimensionality reduction
This work addresses the challenge of high-dimensional latent spaces in normalizing flows for machine learning practitioners, though it appears incremental as it builds on the SurVAE framework.
The paper tackled the problem of dimensionality reduction in normalizing flows by introducing a new layer called the funnel, which enables exact maximum likelihood training with reduced latent space size. The result showed that this approach improves or matches the performance of existing flows on various datasets.
Normalizing flows are diffeomorphic, typically dimension-preserving, models trained using the likelihood of the model. We use the SurVAE framework to construct dimension reducing surjective flows via a new layer, known as the funnel. We demonstrate its efficacy on a variety of datasets, and show it improves upon or matches the performance of existing flows while having a reduced latent space size. The funnel layer can be constructed from a wide range of transformations including restricted convolution and feed forward layers.