Drivable Avatar Clothing: Faithful Full-Body Telepresence with Dynamic Clothing Driven by Sparse RGB-D Input
This work addresses the problem of realistic clothing simulation in telepresence avatars for applications like virtual reality, though it appears incremental as it builds on existing avatar and tracking techniques.
The paper tackles the challenge of modeling dynamically moving loose clothing in photorealistic avatars by proposing a method that uses sparse RGB-D inputs and motion data to faithfully reconstruct clothing appearance and dynamics, achieving high-fidelity results that generalize to novel environments.
Clothing is an important part of human appearance but challenging to model in photorealistic avatars. In this work we present avatars with dynamically moving loose clothing that can be faithfully driven by sparse RGB-D inputs as well as body and face motion. We propose a Neural Iterative Closest Point (N-ICP) algorithm that can efficiently track the coarse garment shape given sparse depth input. Given the coarse tracking results, the input RGB-D images are then remapped to texel-aligned features, which are fed into the drivable avatar models to faithfully reconstruct appearance details. We evaluate our method against recent image-driven synthesis baselines, and conduct a comprehensive analysis of the N-ICP algorithm. We demonstrate that our method can generalize to a novel testing environment, while preserving the ability to produce high-fidelity and faithful clothing dynamics and appearance.