PRISM: Probabilistic Representation for Integrated Shape Modeling and Generation
This work addresses the problem of generating high-fidelity and structurally coherent 3D shapes with part-level control for applications in computer graphics and design, representing a novel method for a known bottleneck rather than a foundational advancement.
The paper tackles the challenge of accurately modeling complex geometries and semantics of shape parts in 3D shape generation, particularly for shapes with varying numbers of parts, by introducing PRISM, a compositional approach that integrates categorical diffusion models with Statistical Shape Models and Gaussian Mixture Models, resulting in significant outperformance in quality and controllability over previous methods.
Despite the advancements in 3D full-shape generation, accurately modeling complex geometries and semantics of shape parts remains a significant challenge, particularly for shapes with varying numbers of parts. Current methods struggle to effectively integrate the contextual and structural information of 3D shapes into their generative processes. We address these limitations with PRISM, a novel compositional approach for 3D shape generation that integrates categorical diffusion models with Statistical Shape Models (SSM) and Gaussian Mixture Models (GMM). Our method employs compositional SSMs to capture part-level geometric variations and uses GMM to represent part semantics in a continuous space. This integration enables both high fidelity and diversity in generated shapes while preserving structural coherence. Through extensive experiments on shape generation and manipulation tasks, we demonstrate that our approach significantly outperforms previous methods in both quality and controllability of part-level operations. Our code will be made publicly available.