ViewFusion: Towards Multi-View Consistency via Interpolated Denoising
This addresses the challenge of generating consistent multiple views for applications like 3D reconstruction or virtual reality, representing an incremental improvement by extending existing models.
The paper tackles the problem of maintaining multi-view consistency in novel-view synthesis with diffusion models, introducing ViewFusion, a training-free algorithm that integrates into pre-trained models to achieve robust consistency without fine-tuning.
Novel-view synthesis through diffusion models has demonstrated remarkable potential for generating diverse and high-quality images. Yet, the independent process of image generation in these prevailing methods leads to challenges in maintaining multiple-view consistency. To address this, we introduce ViewFusion, a novel, training-free algorithm that can be seamlessly integrated into existing pre-trained diffusion models. Our approach adopts an auto-regressive method that implicitly leverages previously generated views as context for the next view generation, ensuring robust multi-view consistency during the novel-view generation process. Through a diffusion process that fuses known-view information via interpolated denoising, our framework successfully extends single-view conditioned models to work in multiple-view conditional settings without any additional fine-tuning. Extensive experimental results demonstrate the effectiveness of ViewFusion in generating consistent and detailed novel views.