VR-NeRF: High-Fidelity Virtualized Walkable Spaces
This work addresses the challenge of creating immersive VR environments with high visual quality, though it appears incremental as it builds on existing neural radiance field methods.
The authors tackled the problem of capturing and rendering walkable spaces in virtual reality with high fidelity by developing an end-to-end system using neural radiance fields, achieving real-time rendering at 36 Hz on dual 2K×2K resolution. They extended instant neural graphics primitives with a perceptual color space and mip-mapping for improved quality and speed.
We present an end-to-end system for the high-fidelity capture, model reconstruction, and real-time rendering of walkable spaces in virtual reality using neural radiance fields. To this end, we designed and built a custom multi-camera rig to densely capture walkable spaces in high fidelity and with multi-view high dynamic range images in unprecedented quality and density. We extend instant neural graphics primitives with a novel perceptual color space for learning accurate HDR appearance, and an efficient mip-mapping mechanism for level-of-detail rendering with anti-aliasing, while carefully optimizing the trade-off between quality and speed. Our multi-GPU renderer enables high-fidelity volume rendering of our neural radiance field model at the full VR resolution of dual 2K$\times$2K at 36 Hz on our custom demo machine. We demonstrate the quality of our results on our challenging high-fidelity datasets, and compare our method and datasets to existing baselines. We release our dataset on our project website.