FoVolNet: Fast Volume Rendering using Foveated Deep Neural Networks
This work addresses the problem of slow volume rendering for scientific and engineering visualization, offering a performance boost for applications such as virtual reality, though it is incremental as it builds on existing foveated rendering and neural reconstruction techniques.
The paper tackles the challenge of achieving high-quality, interactive volume rendering for demanding applications like virtual reality by introducing FoVolNet, a foveated deep neural network method that sparsely samples volume data and reconstructs full frames, resulting in significant time savings over conventional rendering while preserving perceptual quality.
Volume data is found in many important scientific and engineering applications. Rendering this data for visualization at high quality and interactive rates for demanding applications such as virtual reality is still not easily achievable even using professional-grade hardware. We introduce FoVolNet -- a method to significantly increase the performance of volume data visualization. We develop a cost-effective foveated rendering pipeline that sparsely samples a volume around a focal point and reconstructs the full-frame using a deep neural network. Foveated rendering is a technique that prioritizes rendering computations around the user's focal point. This approach leverages properties of the human visual system, thereby saving computational resources when rendering data in the periphery of the user's field of vision. Our reconstruction network combines direct and kernel prediction methods to produce fast, stable, and perceptually convincing output. With a slim design and the use of quantization, our method outperforms state-of-the-art neural reconstruction techniques in both end-to-end frame times and visual quality. We conduct extensive evaluations of the system's rendering performance, inference speed, and perceptual properties, and we provide comparisons to competing neural image reconstruction techniques. Our test results show that FoVolNet consistently achieves significant time saving over conventional rendering while preserving perceptual quality.