Gen3DSR: Generalizable 3D Scene Reconstruction via Divide and Conquer from a Single View
This addresses the challenge of reconstructing diverse 3D scenes from single images for applications in robotics or AR/VR, though it appears incremental as it combines existing holistic and object-level approaches.
The paper tackles the problem of single-view 3D scene reconstruction, which struggles with real-world complexity, by proposing a hybrid divide-and-conquer method that generalizes to various scenes without retraining, achieving favorable performance compared to prior works.
Single-view 3D reconstruction is currently approached from two dominant perspectives: reconstruction of scenes with limited diversity using 3D data supervision or reconstruction of diverse singular objects using large image priors. However, real-world scenarios are far more complex and exceed the capabilities of these methods. We therefore propose a hybrid method following a divide-and-conquer strategy. We first process the scene holistically, extracting depth and semantic information, and then leverage an object-level method for the detailed reconstruction of individual components. By splitting the problem into simpler tasks, our system is able to generalize to various types of scenes without retraining or fine-tuning. We purposely design our pipeline to be highly modular with independent, self-contained modules, to avoid the need for end-to-end training of the whole system. This enables the pipeline to naturally improve as future methods can replace the individual modules. We demonstrate the reconstruction performance of our approach on both synthetic and real-world scenes, comparing favorable against prior works. Project page: https://andreeadogaru.github.io/Gen3DSR