ET3D: Efficient Text-to-3D Generation via Multi-View Distillation
This addresses the efficiency problem for users and service providers in 3D content creation, though it is incremental as it builds on existing distillation paradigms.
The paper tackles the slow generation speed in text-to-3D generation by introducing a method that reduces the time to around 8 ms per 3D asset on a consumer GPU, using multi-view distillation from a pre-trained text-to-image diffusion model to train a 3D GAN.
Recent breakthroughs in text-to-image generation has shown encouraging results via large generative models. Due to the scarcity of 3D assets, it is hardly to transfer the success of text-to-image generation to that of text-to-3D generation. Existing text-to-3D generation methods usually adopt the paradigm of DreamFusion, which conducts per-asset optimization by distilling a pretrained text-to-image diffusion model. The generation speed usually ranges from several minutes to tens of minutes per 3D asset, which degrades the user experience and also imposes a burden to the service providers due to the high computational budget. In this work, we present an efficient text-to-3D generation method, which requires only around 8 $ms$ to generate a 3D asset given the text prompt on a consumer graphic card. The main insight is that we exploit the images generated by a large pre-trained text-to-image diffusion model, to supervise the training of a text conditioned 3D generative adversarial network. Once the network is trained, we are able to efficiently generate a 3D asset via a single forward pass. Our method requires no 3D training data and provides an alternative approach for efficient text-to-3D generation by distilling pre-trained image diffusion models.