GET3D--: Learning GET3D from Unconstrained Image Collections
This addresses a bottleneck in 3D industries by enabling efficient 3D model generation from unconstrained image collections, though it is incremental as it builds on existing generative models.
The paper tackles the problem of generating 3D textured shapes from 2D images with unknown camera pose and scale, proposing GET3D-- which learns a camera sampler and shape generator to produce high-quality shapes on synthetic and realistic datasets.
The demand for efficient 3D model generation techniques has grown exponentially, as manual creation of 3D models is time-consuming and requires specialized expertise. While generative models have shown potential in creating 3D textured shapes from 2D images, their applicability in 3D industries is limited due to the lack of a well-defined camera distribution in real-world scenarios, resulting in low-quality shapes. To overcome this limitation, we propose GET3D--, the first method that directly generates textured 3D shapes from 2D images with unknown pose and scale. GET3D-- comprises a 3D shape generator and a learnable camera sampler that captures the 6D external changes on the camera. In addition, We propose a novel training schedule to stably optimize both the shape generator and camera sampler in a unified framework. By controlling external variations using the learnable camera sampler, our method can generate aligned shapes with clear textures. Extensive experiments demonstrate the efficacy of GET3D--, which precisely fits the 6D camera pose distribution and generates high-quality shapes on both synthetic and realistic unconstrained datasets.