Real-time volumetric rendering of dynamic humans
This enables efficient and interactive visualization of dynamic humans for applications like VR, but it is incremental as it builds on existing parametric body fits and neural representations.
The paper tackles the problem of fast 3D reconstruction and real-time rendering of dynamic humans from monocular videos, achieving reconstruction in less than 3 hours on a single GPU (compared to up to 72 hours for state-of-the-art methods) and rendering at 40 frames per second on a mobile VR device.
We present a method for fast 3D reconstruction and real-time rendering of dynamic humans from monocular videos with accompanying parametric body fits. Our method can reconstruct a dynamic human in less than 3h using a single GPU, compared to recent state-of-the-art alternatives that take up to 72h. These speedups are obtained by using a lightweight deformation model solely based on linear blend skinning, and an efficient factorized volumetric representation for modeling the shape and color of the person in canonical pose. Moreover, we propose a novel local ray marching rendering which, by exploiting standard GPU hardware and without any baking or conversion of the radiance field, allows visualizing the neural human on a mobile VR device at 40 frames per second with minimal loss of visual quality. Our experimental evaluation shows superior or competitive results with state-of-the art methods while obtaining large training speedup, using a simple model, and achieving real-time rendering.