PanoLora: Bridging Perspective and Panoramic Video Generation with LoRA Adaptation
This addresses the problem of inefficient and suboptimal panoramic video generation for applications requiring immersive 360° content, representing a domain-specific incremental improvement.
The paper tackles the challenge of generating high-quality 360° panoramic videos by treating it as an adaptation problem from perspective views using Low-Rank Adaptation (LoRA), achieving superior visual quality, left-right consistency, and motion diversity compared to previous state-of-the-art methods while fine-tuning with only about 1,000 videos.
Generating high-quality 360° panoramic videos remains a significant challenge due to the fundamental differences between panoramic and traditional perspective-view projections. While perspective videos rely on a single viewpoint with a limited field of view, panoramic content requires rendering the full surrounding environment, making it difficult for standard video generation models to adapt. Existing solutions often introduce complex architectures or large-scale training, leading to inefficiency and suboptimal results. Motivated by the success of Low-Rank Adaptation (LoRA) in style transfer tasks, we propose treating panoramic video generation as an adaptation problem from perspective views. Through theoretical analysis, we demonstrate that LoRA can effectively model the transformation between these projections when its rank exceeds the degrees of freedom in the task. Our approach efficiently fine-tunes a pretrained video diffusion model using only approximately 1,000 videos while achieving high-quality panoramic generation. Experimental results demonstrate that our method maintains proper projection geometry and surpasses previous state-of-the-art approaches in visual quality, left-right consistency, and motion diversity.