JointRF: End-to-End Joint Optimization for Dynamic Neural Radiance Field Representation and Compression
This work addresses the problem of high data requirements for volumetric videos in computer vision, offering a domain-specific incremental improvement.
The paper tackles the challenge of rendering dynamic and long-sequence neural radiance fields by proposing JointRF, an end-to-end joint optimization scheme for representation and compression, achieving significantly improved quality and compression efficiency over previous methods.
Neural Radiance Field (NeRF) excels in photo-realistically static scenes, inspiring numerous efforts to facilitate volumetric videos. However, rendering dynamic and long-sequence radiance fields remains challenging due to the significant data required to represent volumetric videos. In this paper, we propose a novel end-to-end joint optimization scheme of dynamic NeRF representation and compression, called JointRF, thus achieving significantly improved quality and compression efficiency against the previous methods. Specifically, JointRF employs a compact residual feature grid and a coefficient feature grid to represent the dynamic NeRF. This representation handles large motions without compromising quality while concurrently diminishing temporal redundancy. We also introduce a sequential feature compression subnetwork to further reduce spatial-temporal redundancy. Finally, the representation and compression subnetworks are end-to-end trained combined within the JointRF. Extensive experiments demonstrate that JointRF can achieve superior compression performance across various datasets.