Mind the Gap: Geometrically Accurate Generative Reconstruction from Disjoint Views
This work addresses a fundamental limitation of 3D vision systems—the need for visual overlap—which is critical for applications like swarm robotics and crowd-sourced data collection.
The paper introduces Generative Reconstruction from Disjoint Views, a new paradigm for 3D reconstruction from non-overlapping images, and proposes GLADOS, a modular framework that uses generative models to bridge gaps and achieve geometrically accurate reconstructions. GLADOS significantly outperforms existing methods, which fail catastrophically on this task.
3D vision systems are fundamentally constrained by their reliance on visual overlap: reconstruction methods require it for geometric alignment, while generative models use it to enforce multi-view consistency. This limitation is particularly acute in real-world scenarios such as distributed swarm robotics or crowd-sourced data collection, where capturing overlapping perspectives, both in terms of spatial and appearance overlap, is often impossible. We introduce Generative Reconstruction from Disjoint Views as a new paradigm, establish a comprehensive dataset, and propose specialized evaluation metrics for zero-overlap scenarios. Our benchmarking demonstrates that existing state-of-the-art methods fail catastrophically on this task, producing disconnected geometries or semantically incoherent reconstructions. To address these limitations, we propose GLADOS, a general, modular framework that operates through three stages: (1) Generative Bridging, where foundation models synthesize intermediate perspectives to connect disjoint inputs; (2) Robust Coarse 3D Reconstruction, that establish coarse geometric scaffold via global alignment which absorbs local contradictions from generative process; and (3) Iterative Context Expansion and Consistency Optimization to fill missing regions and unify the reconstruction. As an architectureagnostic framework, GLADOS enables seamless integration of future advances in generation, reconstruction, and inpainting. The source code is available at: https://github.com/gwilczynski95/GLADOS.