Training Normalizing Flows from Dependent Data
This addresses a practical limitation in normalizing flows for researchers and practitioners dealing with dependent data, but it is incremental as it builds on existing methods by adding dependency handling.
The paper tackled the problem that normalizing flows assume independent data, which is often violated in practice, leading to errors in density estimation and generation; they proposed a likelihood objective incorporating dependencies, showing improved results on synthetic and real-world data, including higher statistical power in genome-wide association studies.
Normalizing flows are powerful non-parametric statistical models that function as a hybrid between density estimators and generative models. Current learning algorithms for normalizing flows assume that data points are sampled independently, an assumption that is frequently violated in practice, which may lead to erroneous density estimation and data generation. We propose a likelihood objective of normalizing flows incorporating dependencies between the data points, for which we derive a flexible and efficient learning algorithm suitable for different dependency structures. We show that respecting dependencies between observations can improve empirical results on both synthetic and real-world data, and leads to higher statistical power in a downstream application to genome-wide association studies.