Sparc3D: Sparse Representation and Construction for High-Resolution 3D Shapes Modeling
This work addresses the problem of inefficient 3D shape modeling for applications like computer graphics and AI, offering a scalable solution for high-resolution generation, though it appears incremental by building on existing latent diffusion methods.
The paper tackles the challenge of high-fidelity 3D object synthesis by introducing Sparc3D, a unified framework that combines a sparse deformable marching cubes representation and a novel sparse convolutional VAE, achieving state-of-the-art reconstruction fidelity on complex geometries while reducing computational costs.
High-fidelity 3D object synthesis remains significantly more challenging than 2D image generation due to the unstructured nature of mesh data and the cubic complexity of dense volumetric grids. Existing two-stage pipelines-compressing meshes with a VAE (using either 2D or 3D supervision), followed by latent diffusion sampling-often suffer from severe detail loss caused by inefficient representations and modality mismatches introduced in VAE. We introduce Sparc3D, a unified framework that combines a sparse deformable marching cubes representation Sparcubes with a novel encoder Sparconv-VAE. Sparcubes converts raw meshes into high-resolution ($1024^3$) surfaces with arbitrary topology by scattering signed distance and deformation fields onto a sparse cube, allowing differentiable optimization. Sparconv-VAE is the first modality-consistent variational autoencoder built entirely upon sparse convolutional networks, enabling efficient and near-lossless 3D reconstruction suitable for high-resolution generative modeling through latent diffusion. Sparc3D achieves state-of-the-art reconstruction fidelity on challenging inputs, including open surfaces, disconnected components, and intricate geometry. It preserves fine-grained shape details, reduces training and inference cost, and integrates naturally with latent diffusion models for scalable, high-resolution 3D generation.