A Comparison of PDF Projection with Normalizing Flows and SurVAE
This work clarifies the historical context for researchers in generative modeling, showing that recent advances are incremental rather than novel.
The paper reveals that recent generative network methods, Normalizing Flows and SurVAE, are re-inventions of the older PDF projection technique, which has been more extensively developed over the past two decades.
Normalizing flows (NF) recently gained attention as a way to construct generative networks with exact likelihood calculation out of composable layers. However, NF is restricted to dimension-preserving transformations. Surjection VAE (SurVAE) has been proposed to extend NF to dimension-altering transformations. Such networks are desirable because they are expressive and can be precisely trained. We show that the approaches are a re-invention of PDF projection, which appeared over twenty years earlier and is much further developed.