An End-to-End Network for Upright Adjustment of Panoramic Images
This work addresses the issue of non-upright panoramic images for users of panoramic cameras, but it is incremental as it builds on existing methods by integrating angle estimation and reconstruction into a single online process.
The paper tackles the problem of upright adjustment for tilted panoramic images by proposing an end-to-end network that estimates the tilt angle and reconstructs the image in real-time, achieving improved accuracy for small angle errors and enabling the first real-time online upright reconstruction using deep learning.
Nowadays, panoramic images can be easily obtained by panoramic cameras. However, when the panoramic camera orientation is tilted, a non-upright panoramic image will be captured. Existing upright adjustment models focus on how to estimate more accurate camera orientation, and attribute image reconstruction to offline or post-processing tasks. To this end, we propose an online end-to-end network for upright adjustment. Our network is designed to reconstruct the image while finding the angle. Our network consists of three modules: orientation estimation, LUT online generation, and upright reconstruction. Direction estimation estimates the tilt angle of the panoramic image. Then, a converter block with upsampling function is designed to generate angle to LUT. This module can output corresponding online LUT for different input angles. Finally, a lightweight generative adversarial network (GAN) aims to generate upright images from shallow features. The experimental results show that in terms of angles, we have improved the accuracy of small angle errors. In terms of image reconstruction, In image reconstruction, we have achieved the first real-time online upright reconstruction of panoramic images using deep learning networks.