Generative Panoramic Image Stitching
This addresses the challenge of creating high-quality panoramas for photography and computer vision applications, but it is incremental as it builds on existing generative models.
The paper tackles the problem of generating seamless panoramas from multiple reference images with parallax and lighting variations, where traditional stitching fails and generative models struggle with large coherent regions. The proposed method fine-tunes a diffusion-based inpainting model to preserve scene content and layout, outperforming baselines in image quality and consistency on captured datasets.
We introduce the task of generative panoramic image stitching, which aims to synthesize seamless panoramas that are faithful to the content of multiple reference images containing parallax effects and strong variations in lighting, camera capture settings, or style. In this challenging setting, traditional image stitching pipelines fail, producing outputs with ghosting and other artifacts. While recent generative models are capable of outpainting content consistent with multiple reference images, they fail when tasked with synthesizing large, coherent regions of a panorama. To address these limitations, we propose a method that fine-tunes a diffusion-based inpainting model to preserve a scene's content and layout based on multiple reference images. Once fine-tuned, the model outpaints a full panorama from a single reference image, producing a seamless and visually coherent result that faithfully integrates content from all reference images. Our approach significantly outperforms baselines for this task in terms of image quality and the consistency of image structure and scene layout when evaluated on captured datasets.