EfficientDreamer: High-Fidelity and Robust 3D Creation via Orthogonal-view Diffusion Prior
This addresses a specific bottleneck in 3D generation for applications like graphics and design, offering a robust solution to view inconsistency issues.
The paper tackles the Janus problem in text-driven 3D content creation, where diffusion models often generate multi-faced models due to inaccurate view information, by proposing a pipeline that uses orthogonal-view diffusion priors to improve 3D consistency and quality, resulting in surpassing previous text-to-3D techniques in evaluations.
While image diffusion models have made significant progress in text-driven 3D content creation, they often fail to accurately capture the intended meaning of text prompts, especially for view information. This limitation leads to the Janus problem, where multi-faced 3D models are generated under the guidance of such diffusion models. In this paper, we propose a robust high-quality 3D content generation pipeline by exploiting orthogonal-view image guidance. First, we introduce a novel 2D diffusion model that generates an image consisting of four orthogonal-view sub-images based on the given text prompt. Then, the 3D content is created using this diffusion model. Notably, the generated orthogonal-view image provides strong geometric structure priors and thus improves 3D consistency. As a result, it effectively resolves the Janus problem and significantly enhances the quality of 3D content creation. Additionally, we present a 3D synthesis fusion network that can further improve the details of the generated 3D contents. Both quantitative and qualitative evaluations demonstrate that our method surpasses previous text-to-3D techniques. Project page: https://efficientdreamer.github.io.