On Flow Matching KL Divergence
This work provides theoretical guarantees for flow matching in distribution estimation, making it competitive with diffusion models, which is significant for researchers in generative modeling and statistical machine learning, though it is incremental as it builds on existing flow-matching methods.
The authors derived a deterministic, non-asymptotic upper bound on the KL divergence for flow-matching distribution approximations, showing that if the L2 flow-matching loss is bounded by ε², then the KL divergence is bounded by A₁ε + A₂ε², with constants depending on data and velocity field regularities. This implies statistical convergence rates for Flow Matching Transformers under Total Variation distance, achieving nearly minimax-optimal efficiency comparable to diffusion models.
We derive a deterministic, non-asymptotic upper bound on the Kullback-Leibler (KL) divergence of the flow-matching distribution approximation. In particular, if the $L_2$ flow-matching loss is bounded by $ε^2 > 0$, then the KL divergence between the true data distribution and the estimated distribution is bounded by $A_1 ε+ A_2 ε^2$. Here, the constants $A_1$ and $A_2$ depend only on the regularities of the data and velocity fields. Consequently, this bound implies statistical convergence rates of Flow Matching Transformers under the Total Variation (TV) distance. We show that, flow matching achieves nearly minimax-optimal efficiency in estimating smooth distributions. Our results make the statistical efficiency of flow matching comparable to that of diffusion models under the TV distance. Numerical studies on synthetic and learned velocities corroborate our theory.