IVGT: Implicit Visual Geometry Transformer for Neural Scene Representation
This work addresses the challenge of reconstructing coherent 3D geometry from unposed images, offering a unified implicit representation that generalizes across scenes and tasks.
IVGT introduces an implicit visual geometry transformer that learns continuous 3D geometry and appearance from unposed multi-view images, enabling direct extraction of surfaces and rendering from arbitrary viewpoints. It achieves strong performance across tasks like mesh reconstruction, novel view synthesis, and camera pose estimation.
Reconstructing coherent 3D geometry and appearance from unposed multi-view images is a fundamental yet challenging problem in computer vision. Most existing visual geometry foundation models predict explicit geometry by regressing pixel-aligned pointmaps, often suffering from redundancy and limited geometric continuity. We propose IVGT, an Implicit Visual Geometry Transformer that implicitly models continuous and coherent geometry from pose-free multi-view images. This formulation learns a continuous neural scene representation in a canonical coordinate system and supports continuous spatial queries at any 3D positions, retrieving local features to predict signed distance (SDF) values and colors using lightweight decoders. It allows direct extraction of continuous and coherent surface geometry, enabling rendering of RGB images, depth maps, and surface normal maps from arbitrary viewpoints. We train IVGT via multi-dataset joint optimization with 2D supervision and 3D geometric regularization. IVGT demonstrates generalization across scenes and achieves strong performance on various tasks, including mesh and point cloud reconstruction, novel view synthesis, depth and surface normal estimation, and camera pose estimation.