Casual 6-DoF: free-viewpoint panorama using a handheld 360 camera
This work addresses the need for easier and faster 6-DoF video capture for casual users, representing an incremental improvement over existing image-based rendering methods.
The paper tackles the problem of creating six degrees-of-freedom (6-DoF) video from 360° panoramas to enable free-viewpoint movement in captured scenes, achieving better visual quality and robustness compared to state-of-the-art methods in various environments.
Six degrees-of-freedom (6-DoF) video provides telepresence by enabling users to move around in the captured scene with a wide field of regard. Compared to methods requiring sophisticated camera setups, the image-based rendering method based on photogrammetry can work with images captured with any poses, which is more suitable for casual users. However, existing image-based rendering methods are based on perspective images. When used to reconstruct 6-DoF views, it often requires capturing hundreds of images, making data capture a tedious and time-consuming process. In contrast to traditional perspective images, 360° images capture the entire surrounding view in a single shot, thus, providing a faster capturing process for 6-DoF view reconstruction. This paper presents a novel method to provide 6-DoF experiences over a wide area using an unstructured collection of 360° panoramas captured by a conventional 360° camera. Our method consists of 360° data capturing, novel depth estimation to produce a high-quality spherical depth panorama, and high-fidelity free-viewpoint generation. We compared our method against state-of-the-art methods, using data captured in various environments. Our method shows better visual quality and robustness in the tested scenes.