NeRV360: Neural Representation for 360-Degree Videos with a Viewport Decoder
This addresses the challenge of real-time applications for 360-degree video compression, offering a domain-specific incremental improvement.
The paper tackled the problem of high memory usage and slow decoding in applying neural implicit representations to 360-degree videos by proposing NeRV360, which decodes only the user-selected viewport, resulting in a 7-fold reduction in memory consumption and a 2.5-fold increase in decoding speed compared to prior work while improving image quality.
Implicit neural representations for videos (NeRV) have shown strong potential for video compression. However, applying NeRV to high-resolution 360-degree videos causes high memory usage and slow decoding, making real-time applications impractical. We propose NeRV360, an end-to-end framework that decodes only the user-selected viewport instead of reconstructing the entire panoramic frame. Unlike conventional pipelines, NeRV360 integrates viewport extraction into decoding and introduces a spatial-temporal affine transform module for conditional decoding based on viewpoint and time. Experiments on 6K-resolution videos show that NeRV360 achieves a 7-fold reduction in memory consumption and a 2.5-fold increase in decoding speed compared to HNeRV, a representative prior work, while delivering better image quality in terms of objective metrics.