DreamFlow: High-Quality Text-to-3D Generation by Approximating Probability Flow
This work addresses efficiency and quality issues in text-to-3D generation for applications like content creation, but it is incremental as it builds on existing score distillation methods.
The paper tackles the problem of slow and high-variance optimization in text-to-3D generation by proposing DreamFlow, a framework that approximates probability flow with a predetermined timestep schedule, resulting in 5 times faster generation than state-of-the-art methods while producing more photorealistic 3D content at 1024x1024 resolution.
Recent progress in text-to-3D generation has been achieved through the utilization of score distillation methods: they make use of the pre-trained text-to-image (T2I) diffusion models by distilling via the diffusion model training objective. However, such an approach inevitably results in the use of random timesteps at each update, which increases the variance of the gradient and ultimately prolongs the optimization process. In this paper, we propose to enhance the text-to-3D optimization by leveraging the T2I diffusion prior in the generative sampling process with a predetermined timestep schedule. To this end, we interpret text-to3D optimization as a multi-view image-to-image translation problem, and propose a solution by approximating the probability flow. By leveraging the proposed novel optimization algorithm, we design DreamFlow, a practical three-stage coarseto-fine text-to-3D optimization framework that enables fast generation of highquality and high-resolution (i.e., 1024x1024) 3D contents. For example, we demonstrate that DreamFlow is 5 times faster than the existing state-of-the-art text-to-3D method, while producing more photorealistic 3D contents. Visit our project page (https://kyungmnlee.github.io/dreamflow.github.io/) for visualizations.