SoFlow: Solution Flow Models for One-Step Generative Modeling
This addresses efficiency issues in generative modeling for AI applications, but it is incremental as it builds on existing methods like Flow Matching.
The paper tackles the efficiency problem of multi-step denoising in diffusion models by proposing SoFlow, a framework for one-step generation from scratch, achieving better FID-50K scores than MeanFlow models on ImageNet 256x256.
The multi-step denoising process in diffusion and Flow Matching models causes major efficiency issues, which motivates research on few-step generation. We present Solution Flow Models (SoFlow), a framework for one-step generation from scratch. By analyzing the relationship between the velocity function and the solution function of the velocity ordinary differential equation (ODE), we propose a Flow Matching loss and a solution consistency loss to train our models. The Flow Matching loss allows our models to provide estimated velocity fields for Classifier-Free Guidance (CFG) during training, which improves generation performance. Notably, our consistency loss does not require the calculation of the Jacobian-vector product (JVP), a common requirement in recent works that is not well-optimized in deep learning frameworks like PyTorch. Experimental results indicate that, when trained from scratch using the same Diffusion Transformer (DiT) architecture and an equal number of training epochs, our models achieve better FID-50K scores than MeanFlow models on the ImageNet 256x256 dataset.