Simplifying Flow Matching Transformations with Low-Rank Mixture Models
For practitioners of normalizing flows, this work offers a practical way to simplify flow transformations by better aligning latent and data distributions, though the gains are incremental over existing methods.
The authors propose using mixtures of probabilistic principal component analyzers (MPPCA) as the latent density in normalizing flows to reduce topological mismatch, leading to faster training and improved generative performance. On tabular and image datasets, they report consistent gains in training efficiency and generation quality over baselines.
Normalizing flows are powerful generative models that learn an invertible mapping between complex data distributions and simple latent distributions, typically a standard normal density. However, this choice of latent density can impose unnecessary complexity on the learned flow transformation due to the topological mismatch between the latent and data densities, leading to slower training and suboptimal performance. In this work, we propose using mixtures of probabilistic principal component analyzers (MPPCA) as the latent density for normalizing flows. We simplify the learned flow transformation by learning a latent distribution that more closely aligns with the data distribution in terms of KL divergence, thus enabling faster convergence and improved generative performance. Critically, MPPCA models can be fit quickly and cheaply using the expectation-maximization algorithm, making them a practical choice for initializing latent distributions even in high-dimensional generative tasks. We validate our method on both tabular and image datasets, demonstrating consistent gains in training efficiency and generation quality compared to baselines.