Towards Real-Time Neural Volumetric Rendering on Mobile Devices: A Measurement Study
This work addresses the problem of slow NeRF rendering for mobile applications, but it is incremental as it focuses on measurement rather than new methods.
The paper tackles the challenge of enabling real-time neural volumetric rendering on mobile devices by conducting a measurement study on NeRF serving systems, identifying that mesh granularity is the most effective control knob for performance improvement while quantization is the least effective.
Neural Radiance Fields (NeRF) is an emerging technique to synthesize 3D objects from 2D images with a wide range of potential applications. However, rendering existing NeRF models is extremely computation intensive, making it challenging to support real-time interaction on mobile devices. In this paper, we take the first initiative to examine the state-of-the-art real-time NeRF rendering technique from a system perspective. We first define the entire working pipeline of the NeRF serving system. We then identify possible control knobs that are critical to the system from the communication, computation, and visual performance perspective. Furthermore, an extensive measurement study is conducted to reveal the effects of these control knobs on system performance. Our measurement results reveal that different control knobs contribute differently towards improving the system performance, with the mesh granularity being the most effective knob and the quantization being the least effective knob. In addition, diverse hardware device settings and network conditions have to be considered to fully unleash the benefit of operating under the appropriate knobs