Birth and Death of a Rose
This addresses the challenge of automating 3D animation for dynamic objects, reducing manual effort for creators in graphics and simulation.
The paper tackles the problem of generating temporally consistent 3D object intrinsics, such as a blooming rose, from pre-trained 2D diffusion models, achieving high-quality results that allow controllable rendering from any viewpoint and lighting condition.
We study the problem of generating temporal object intrinsics -- temporally evolving sequences of object geometry, reflectance, and texture, such as a blooming rose -- from pre-trained 2D foundation models. Unlike conventional 3D modeling and animation techniques that require extensive manual effort and expertise, we introduce a method that generates such assets with signals distilled from pre-trained 2D diffusion models. To ensure the temporal consistency of object intrinsics, we propose Neural Templates for temporal-state-guided distillation, derived automatically from image features from self-supervised learning. Our method can generate high-quality temporal object intrinsics for several natural phenomena and enable the sampling and controllable rendering of these dynamic objects from any viewpoint, under any environmental lighting conditions, at any time of their lifespan. Project website: https://chen-geng.com/rose4d