4D Temporally Coherent Light-field Video
This work addresses a major challenge in light-field video for virtual and augmented reality applications, enabling more realistic and immersive experiences.
The paper tackles the problem of achieving temporal coherence in light-field videos, which is crucial for storage, compression, and editing, by proposing a method that uses Epipolar Plane Images to constrain scene flow estimation, resulting in a significant improvement in accuracy over existing approaches.
Light-field video has recently been used in virtual and augmented reality applications to increase realism and immersion. However, existing light-field methods are generally limited to static scenes due to the requirement to acquire a dense scene representation. The large amount of data and the absence of methods to infer temporal coherence pose major challenges in storage, compression and editing compared to conventional video. In this paper, we propose the first method to extract a spatio-temporally coherent light-field video representation. A novel method to obtain Epipolar Plane Images (EPIs) from a spare light-field camera array is proposed. EPIs are used to constrain scene flow estimation to obtain 4D temporally coherent representations of dynamic light-fields. Temporal coherence is achieved on a variety of light-field datasets. Evaluation of the proposed light-field scene flow against existing multi-view dense correspondence approaches demonstrates a significant improvement in accuracy of temporal coherence.