DreamDrone: Text-to-Image Diffusion Models are Zero-shot Perpetual View Generators
This provides a zero-shot, training-free solution for perpetual view generation, which is incremental as it builds on existing text-to-image diffusion models.
The paper tackles the problem of generating unbounded flythrough scenes from text prompts without training, by warping latent codes in diffusion models and enhancing consistency, resulting in significantly higher visual quality than existing methods.
We introduce DreamDrone, a novel zero-shot and training-free pipeline for generating unbounded flythrough scenes from textual prompts. Different from other methods that focus on warping images frame by frame, we advocate explicitly warping the intermediate latent code of the pre-trained text-to-image diffusion model for high-quality image generation and generalization ability. To further enhance the fidelity of the generated images, we also propose a feature-correspondence-guidance diffusion process and a high-pass filtering strategy to promote geometric consistency and high-frequency detail consistency, respectively. Extensive experiments reveal that DreamDrone significantly surpasses existing methods, delivering highly authentic scene generation with exceptional visual quality, without training or fine-tuning on datasets or reconstructing 3D point clouds in advance.