HiFi-123: Towards High-fidelity One Image to 3D Content Generation
This work addresses the problem of low-fidelity 3D generation from images for practical applications in computer vision and graphics, representing an incremental advancement over existing methods.
The paper tackles the problem of generating high-fidelity 3D content from a single image, addressing issues like blurred textures and deviations in novel views, and achieves state-of-the-art performance with significant improvements in 3D generation quality.
Recent advances in diffusion models have enabled 3D generation from a single image. However, current methods often produce suboptimal results for novel views, with blurred textures and deviations from the reference image, limiting their practical applications. In this paper, we introduce HiFi-123, a method designed for high-fidelity and multi-view consistent 3D generation. Our contributions are twofold: First, we propose a Reference-Guided Novel View Enhancement (RGNV) technique that significantly improves the fidelity of diffusion-based zero-shot novel view synthesis methods. Second, capitalizing on the RGNV, we present a novel Reference-Guided State Distillation (RGSD) loss. When incorporated into the optimization-based image-to-3D pipeline, our method significantly improves 3D generation quality, achieving state-of-the-art performance. Comprehensive evaluations demonstrate the effectiveness of our approach over existing methods, both qualitatively and quantitatively. Video results are available on the project page.