UniLat3D: Geometry-Appearance Unified Latents for Single-Stage 3D Generation
This addresses the need for efficient and aligned 3D asset generation in industries like gaming and VR, though it appears incremental as it builds on existing latent space and flow-matching methods.
The paper tackles the problem of geometry-texture misalignment and high cost in two-stage 3D generation by proposing UniLat3D, a unified framework that encodes geometry and appearance in a single latent space, enabling direct single-stage generation and producing high-quality 3D assets in seconds from a single image.
High-fidelity 3D asset generation is crucial for various industries. While recent 3D pretrained models show strong capability in producing realistic content, most are built upon diffusion models and follow a two-stage pipeline that first generates geometry and then synthesizes appearance. Such a decoupled design tends to produce geometry-texture misalignment and non-negligible cost. In this paper, we propose UniLat3D, a unified framework that encodes geometry and appearance in a single latent space, enabling direct single-stage generation. Our key contribution is a geometry-appearance Unified VAE, which compresses high-resolution sparse features into a compact latent representation -- UniLat. UniLat integrates structural and visual information into a dense low-resolution latent, which can be efficiently decoded into diverse 3D formats, e.g., 3D Gaussians and meshes. Based on this unified representation, we train a single flow-matching model to map Gaussian noise directly into UniLat, eliminating redundant stages. Trained solely on public datasets, UniLat3D produces high-quality 3D assets in seconds from a single image, achieving superior appearance fidelity and geometric quality. More demos \& code are available at https://unilat3d.github.io/