LoRD: Local 4D Implicit Representation for High-Fidelity Dynamic Human Modeling
This work addresses the challenge of capturing detailed surface deformations in dynamic clothed human modeling, which is important for applications in animation and virtual reality, and represents a novel advancement over existing global 4D methods.
The paper tackles the problem of high-fidelity 4D dynamic human modeling by proposing LoRD, a local 4D implicit representation that enables detailed reconstruction of clothing wrinkles and outperforms state-of-the-art methods in applications like 4D reconstruction from sparse points and non-rigid depth fusion.
Recent progress in 4D implicit representation focuses on globally controlling the shape and motion with low dimensional latent vectors, which is prone to missing surface details and accumulating tracking error. While many deep local representations have shown promising results for 3D shape modeling, their 4D counterpart does not exist yet. In this paper, we fill this blank by proposing a novel Local 4D implicit Representation for Dynamic clothed human, named LoRD, which has the merits of both 4D human modeling and local representation, and enables high-fidelity reconstruction with detailed surface deformations, such as clothing wrinkles. Particularly, our key insight is to encourage the network to learn the latent codes of local part-level representation, capable of explaining the local geometry and temporal deformations. To make the inference at test-time, we first estimate the inner body skeleton motion to track local parts at each time step, and then optimize the latent codes for each part via auto-decoding based on different types of observed data. Extensive experiments demonstrate that the proposed method has strong capability for representing 4D human, and outperforms state-of-the-art methods on practical applications, including 4D reconstruction from sparse points, non-rigid depth fusion, both qualitatively and quantitatively.