Weighted Conditional Flow Matching
This addresses a computational bottleneck in normalizing flow generation for machine learning practitioners, though it appears incremental as it builds on existing CFM and optimal transport methods.
The paper tackles the problem of conditional flow matching (CFM) producing inefficient, non-straight paths that slow down generation, by proposing Weighted Conditional Flow Matching (W-CFM) which weights training pairs with a Gibbs kernel to induce straighter trajectories. The result shows W-CFM achieves comparable or superior sample quality, fidelity, and diversity to baselines while maintaining computational efficiency.
Conditional flow matching (CFM) has emerged as a powerful framework for training continuous normalizing flows due to its computational efficiency and effectiveness. However, standard CFM often produces paths that deviate significantly from straight-line interpolations between prior and target distributions, making generation slower and less accurate due to the need for fine discretization at inference. Recent methods enhance CFM performance by inducing shorter and straighter trajectories but typically rely on computationally expensive mini-batch optimal transport (OT). Drawing insights from entropic optimal transport (EOT), we propose Weighted Conditional Flow Matching (W-CFM), a novel approach that modifies the classical CFM loss by weighting each training pair $(x, y)$ with a Gibbs kernel. We show that this weighting recovers the entropic OT coupling up to some bias in the marginals, and we provide the conditions under which the marginals remain nearly unchanged. Moreover, we establish an equivalence between W-CFM and the minibatch OT method in the large-batch limit, showing how our method overcomes computational and performance bottlenecks linked to batch size. Empirically, we test our method on unconditional generation on various synthetic and real datasets, confirming that W-CFM achieves comparable or superior sample quality, fidelity, and diversity to other alternative baselines while maintaining the computational efficiency of vanilla CFM.