Seeing A 3D World in A Grain of Sand
This addresses the challenge of digitizing common miniature scenes for applications like preservation or visualization, though it is incremental as it builds on existing 3D reconstruction methods.
The paper tackles the problem of 3D reconstruction of miniature scenes with millimeter-sized objects, presenting a snapshot imaging technique using a catadioptric system with mirrors and 3D Gaussian Splatting, achieving state-of-the-art performance on synthetic and real scenes.
We present a snapshot imaging technique for recovering 3D surrounding views of miniature scenes. Due to their intricacy, miniature scenes with objects sized in millimeters are difficult to reconstruct, yet miniatures are common in life and their 3D digitalization is desirable. We design a catadioptric imaging system with a single camera and eight pairs of planar mirrors for snapshot 3D reconstruction from a dollhouse perspective. We place paired mirrors on nested pyramid surfaces for capturing surrounding multi-view images in a single shot. Our mirror design is customizable based on the size of the scene for optimized view coverage. We use the 3D Gaussian Splatting (3DGS) representation for scene reconstruction and novel view synthesis. We overcome the challenge posed by our sparse view input by integrating visual hull-derived depth constraint. Our method demonstrates state-of-the-art performance on a variety of synthetic and real miniature scenes.