Long-LRM: Long-sequence Large Reconstruction Model for Wide-coverage Gaussian Splats
This addresses the need for fast, wide-coverage scene reconstruction in computer vision, offering a significant speed improvement over existing methods.
The paper tackles the problem of instant, high-resolution, 3D Gaussian reconstruction from multiple images by proposing Long-LRM, which achieves reconstruction in 1 second on an A100 GPU with quality comparable to optimization-based methods while providing an 800x speedup and handling input sizes at least 60x larger than previous feed-forward approaches.
We propose Long-LRM, a feed-forward 3D Gaussian reconstruction model for instant, high-resolution, 360° wide-coverage, scene-level reconstruction. Specifically, it takes in 32 input images at a resolution of 960x540 and produces the Gaussian reconstruction in just 1 second on a single A100 GPU. To handle the long sequence of 250K tokens brought by the large input size, Long-LRM features a mixture of the recent Mamba2 blocks and the classical transformer blocks, enhanced by a light-weight token merging module and Gaussian pruning steps that balance between quality and efficiency. We evaluate Long-LRM on the large-scale DL3DV benchmark and Tanks&Temples, demonstrating reconstruction quality comparable to the optimization-based methods while achieving an 800x speedup w.r.t. the optimization-based approaches and an input size at least 60x larger than the previous feed-forward approaches. We conduct extensive ablation studies on our model design choices for both rendering quality and computation efficiency. We also explore Long-LRM's compatibility with other Gaussian variants such as 2D GS, which enhances Long-LRM's ability in geometry reconstruction. Project page: https://arthurhero.github.io/projects/llrm