Compact Real-time Radiance Fields with Neural Codebook
This work addresses storage and efficiency challenges for real-time 3D scene reconstruction, though it is incremental as it builds on existing grid-based methods like Plenoxels.
The paper tackles the problem of high storage and transmission overhead in grid-based neural radiance fields by introducing a compression framework that reduces model size by over 40× while maintaining competitive rendering quality and achieving real-time rendering at 180 fps.
Reconstructing neural radiance fields with explicit volumetric representations, demonstrated by Plenoxels, has shown remarkable advantages on training and rendering efficiency, while grid-based representations typically induce considerable overhead for storage and transmission. In this work, we present a simple and effective framework for pursuing compact radiance fields from the perspective of compression methodology. By exploiting intrinsic properties exhibiting in grid models, a non-uniform compression stem is developed to significantly reduce model complexity and a novel parameterized module, named Neural Codebook, is introduced for better encoding high-frequency details specific to per-scene models via a fast optimization. Our approach can achieve over 40 $\times$ reduction on grid model storage with competitive rendering quality. In addition, the method can achieve real-time rendering speed with 180 fps, realizing significant advantage on storage cost compared to real-time rendering methods.