DecoRec: Decomposed 3D Scene Reconstruction from Single-View Images via Object-Level Diffusion
This work addresses the problem of accurate 3D scene reconstruction from a single image, which is challenging due to lack of large-scale datasets and prior methods' inaccuracies.
DecoRec reconstructs a decomposed 3D scene mesh from a single-view 2D image by leveraging diffusion-based object reconstruction and a refinement pipeline, achieving high-quality geometry and novel view synthesis.
In this paper, we introduce \textit{DecoRec}, a novel system designed to elevate single-view 2D images to a decomposed 3D scene mesh. Current methods for single-view scene reconstruction typically rely on object retrieval or the regression of coarse 3D voxels or surfaces, leading to inaccuracies in capturing the appearance and geometry of the input image. The lack of high-quality large-scale scene-level datasets further complicates direct 3D scene generation from single-view images. To achieve high-quality 3D scene generation from a single-view image, DecoRec takes advantage of recent diffusion-based single-view object reconstruction methods to reconstruct individual objects separately. Subsequently, a refinement pipeline is proposed to effectively merge these reconstructed objects, enhancing appearance and geometry through a differentiable rendering technique and diffusion-guided refinement. Our results demonstrate that DecoRec facilitates high-quality single-view scene reconstruction in both geometry and novel synthesis, offering significant benefits for downstream applications like room interior design.