Learning Knowledge-Rich Sequential Model for Planar Homography Estimation in Aerial Video
This work addresses over-fitting issues in homography estimation for aerial video analysis, which is incremental as it builds on existing learning-based methods by introducing sequential processing and regularization.
The paper tackles the problem of severe over-fitting in learning-based planar homography estimation for aerial videos by proposing an unsupervised sequential model that processes video frames in batches and incorporates spatial-temporal knowledge for regularization. Empirical results show significant improvement over image-based methods, with further performance boosts from the regularization.
This paper presents an unsupervised approach that leverages raw aerial videos to learn to estimate planar homographic transformation between consecutive video frames. Previous learning-based estimators work on pairs of images to estimate their planar homographic transformations but suffer from severe over-fitting issues, especially when applying over aerial videos. To address this concern, we develop a sequential estimator that directly processes a sequence of video frames and estimates their pairwise planar homographic transformations in batches. We also incorporate a set of spatial-temporal knowledge to regularize the learning of such a sequence-to-sequence model. We collect a set of challenging aerial videos and compare the proposed method to the alternative algorithms. Empirical studies suggest that our sequential model achieves significant improvement over alternative image-based methods and the knowledge-rich regularization further boosts our system performance. Our codes and dataset could be found at https://github.com/Paul-LiPu/DeepVideoHomography