ImmersePro: End-to-End Stereo Video Synthesis Via Implicit Disparity Learning
This addresses the need for efficient stereo video synthesis for applications like virtual reality, though it appears incremental as it builds on existing stereo generation techniques.
The paper tackles the problem of transforming single-view videos into stereo videos by introducing ImmersePro, a framework that uses implicit disparity learning to generate stereo pairs without explicit disparity maps, achieving improvements of 11.76% (L1), 6.39% (SSIM), and 5.10% (PSNR) over existing methods.
We introduce \textit{ImmersePro}, an innovative framework specifically designed to transform single-view videos into stereo videos. This framework utilizes a novel dual-branch architecture comprising a disparity branch and a context branch on video data by leveraging spatial-temporal attention mechanisms. \textit{ImmersePro} employs implicit disparity guidance, enabling the generation of stereo pairs from video sequences without the need for explicit disparity maps, thus reducing potential errors associated with disparity estimation models. In addition to the technical advancements, we introduce the YouTube-SBS dataset, a comprehensive collection of 423 stereo videos sourced from YouTube. This dataset is unprecedented in its scale, featuring over 7 million stereo pairs, and is designed to facilitate training and benchmarking of stereo video generation models. Our experiments demonstrate the effectiveness of \textit{ImmersePro} in producing high-quality stereo videos, offering significant improvements over existing methods. Compared to the best competitor stereo-from-mono we quantitatively improve the results by 11.76\% (L1), 6.39\% (SSIM), and 5.10\% (PSNR).