Prompt2NeRF-PIL: Fast NeRF Generation via Pretrained Implicit Latent
This addresses the need for faster and more efficient 3D scene generation from prompts, though it is incremental as it builds on existing NeRF and prompt-based methods.
The paper tackles the problem of slow and complex promptable NeRF generation by introducing Prompt2NeRF-PIL, which generates 3D objects via a single forward pass using a pre-trained implicit latent space, resulting in speedups of 3 to 5 times for existing methods like DreamFusion and Zero-1-to-3.
This paper explores promptable NeRF generation (e.g., text prompt or single image prompt) for direct conditioning and fast generation of NeRF parameters for the underlying 3D scenes, thus undoing complex intermediate steps while providing full 3D generation with conditional control. Unlike previous diffusion-CLIP-based pipelines that involve tedious per-prompt optimizations, Prompt2NeRF-PIL is capable of generating a variety of 3D objects with a single forward pass, leveraging a pre-trained implicit latent space of NeRF parameters. Furthermore, in zero-shot tasks, our experiments demonstrate that the NeRFs produced by our method serve as semantically informative initializations, significantly accelerating the inference process of existing prompt-to-NeRF methods. Specifically, we will show that our approach speeds up the text-to-NeRF model DreamFusion and the 3D reconstruction speed of the image-to-NeRF method Zero-1-to-3 by 3 to 5 times.