Text-to-Image Rectified Flow as Plug-and-Play Priors
This work provides a more efficient and effective alternative to diffusion models for researchers and practitioners in generative AI, though it is incremental as it builds on existing prior concepts.
The paper tackles the problem of using generative models as plug-and-play priors for tasks like text-to-3D generation and image inversion, showing that rectified flow-based methods outperform diffusion models in generation quality and efficiency, with fewer inference steps and superior performance in text-to-3D generation compared to SDS and VSD losses.
Large-scale diffusion models have achieved remarkable performance in generative tasks. Beyond their initial training applications, these models have proven their ability to function as versatile plug-and-play priors. For instance, 2D diffusion models can serve as loss functions to optimize 3D implicit models. Rectified flow, a novel class of generative models, enforces a linear progression from the source to the target distribution and has demonstrated superior performance across various domains. Compared to diffusion-based methods, rectified flow approaches surpass in terms of generation quality and efficiency, requiring fewer inference steps. In this work, we present theoretical and experimental evidence demonstrating that rectified flow based methods offer similar functionalities to diffusion models - they can also serve as effective priors. Besides the generative capabilities of diffusion priors, motivated by the unique time-symmetry properties of rectified flow models, a variant of our method can additionally perform image inversion. Experimentally, our rectified flow-based priors outperform their diffusion counterparts - the SDS and VSD losses - in text-to-3D generation. Our method also displays competitive performance in image inversion and editing.