Sequential Flow Straightening for Generative Modeling
This addresses a key bottleneck in generative modeling for faster and higher-quality synthesis, though it appears incremental as it builds on existing flow-based methods.
The paper tackles the slow sampling speed in continuous-time generative models like diffusion and flow-based models by proposing SeqRF, a method that straightens the probability flow to reduce global truncation error, achieving surpassing results on datasets such as CIFAR-10, CelebA-64x64, and LSUN-Church.
Straightening the probability flow of the continuous-time generative models, such as diffusion models or flow-based models, is the key to fast sampling through the numerical solvers, existing methods learn a linear path by directly generating the probability path the joint distribution between the noise and data distribution. One key reason for the slow sampling speed of the ODE-based solvers that simulate these generative models is the global truncation error of the ODE solver, caused by the high curvature of the ODE trajectory, which explodes the truncation error of the numerical solvers in the low-NFE regime. To address this challenge, We propose a novel method called SeqRF, a learning technique that straightens the probability flow to reduce the global truncation error and hence enable acceleration of sampling and improve the synthesis quality. In both theoretical and empirical studies, we first observe the straightening property of our SeqRF. Through empirical evaluations via SeqRF over flow-based generative models, We achieve surpassing results on CIFAR-10, CelebA-$64 \times 64$, and LSUN-Church datasets.