Video Stitching for Linear Camera Arrays
This work addresses video stitching issues for applications like virtual reality and autonomous driving, representing a novel method for a known bottleneck.
The paper tackles the problem of video stitching for linear camera arrays, which often produces artifacts, by proposing a novel algorithm that learns smooth spatial interpolation between input videos, resulting in temporally stable stitching with strong parallax tolerance and outperforming state-of-the-art methods as shown in a user study.
Despite the long history of image and video stitching research, existing academic and commercial solutions still produce strong artifacts. In this work, we propose a wide-baseline video stitching algorithm for linear camera arrays that is temporally stable and tolerant to strong parallax. Our key insight is that stitching can be cast as a problem of learning a smooth spatial interpolation between the input videos. To solve this problem, inspired by pushbroom cameras, we introduce a fast pushbroom interpolation layer and propose a novel pushbroom stitching network, which learns a dense flow field to smoothly align the multiple input videos for spatial interpolation. Our approach outperforms the state-of-the-art by a significant margin, as we show with a user study, and has immediate applications in many areas such as virtual reality, immersive telepresence, autonomous driving, and video surveillance.