FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Views
This addresses a challenging yet practical setting for real-world applications like 3D reconstruction from limited images, though it appears incremental as it builds on existing paradigms with a cascaded learning approach.
The paper tackles the problem of estimating camera poses and 3D geometry from uncalibrated sparse-view images (2-8 inputs) by introducing FLARE, a feed-forward model that achieves state-of-the-art performance in pose estimation, geometry reconstruction, and novel-view synthesis with inference times under 0.5 seconds.
We present FLARE, a feed-forward model designed to infer high-quality camera poses and 3D geometry from uncalibrated sparse-view images (i.e., as few as 2-8 inputs), which is a challenging yet practical setting in real-world applications. Our solution features a cascaded learning paradigm with camera pose serving as the critical bridge, recognizing its essential role in mapping 3D structures onto 2D image planes. Concretely, FLARE starts with camera pose estimation, whose results condition the subsequent learning of geometric structure and appearance, optimized through the objectives of geometry reconstruction and novel-view synthesis. Utilizing large-scale public datasets for training, our method delivers state-of-the-art performance in the tasks of pose estimation, geometry reconstruction, and novel view synthesis, while maintaining the inference efficiency (i.e., less than 0.5 seconds). The project page and code can be found at: https://zhanghe3z.github.io/FLARE/