SOPHY: Learning to Generate Simulation-Ready Objects with Physical Materials
This enables text-driven generation and single-image reconstruction of physics-aware 3D objects for interactive, dynamic environments, representing a novel integration beyond existing static or physics-agnostic models.
The authors tackled the problem of generating 3D objects with physical material properties for simulations, developing SOPHY, a generative model that jointly synthesizes shape, texture, and material attributes, which improved realism and fidelity in experiments.
We present SOPHY, a generative model for 3D physics-aware shape synthesis. Unlike existing 3D generative models that focus solely on static geometry or 4D models that produce physics-agnostic animations, our method jointly synthesizes shape, texture, and material properties related to physics-grounded dynamics, making the generated objects ready for simulations and interactive, dynamic environments. To train our model, we introduce a dataset of 3D objects annotated with detailed physical material attributes, along with an efficient pipeline for material annotation. Our method enables applications such as text-driven generation of interactive, physics-aware 3D objects and single-image reconstruction of physically plausible shapes. Furthermore, our experiments show that jointly modeling shape and material properties enhances the realism and fidelity of the generated shapes, improving performance on both generative geometry and physical plausibility.