PBDyG: Position Based Dynamic Gaussians for Motion-Aware Clothed Human Avatars
This addresses the problem of creating realistic and motion-aware digital avatars for applications in virtual reality, gaming, and animation, representing a novel method for a known bottleneck in avatar reconstruction.
The paper tackles the problem of reconstructing clothed human avatars from multiview RGB videos by introducing a method that recovers physically accurate body and cloth movements, enabling reconstruction of highly deformable garments like skirts or coats, which were challenging with existing methods.
This paper introduces a novel clothed human model that can be learned from multiview RGB videos, with a particular emphasis on recovering physically accurate body and cloth movements. Our method, Position Based Dynamic Gaussians (PBDyG), realizes ``movement-dependent'' cloth deformation via physical simulation, rather than merely relying on ``pose-dependent'' rigid transformations. We model the clothed human holistically but with two distinct physical entities in contact: clothing modeled as 3D Gaussians, which are attached to a skinned SMPL body that follows the movement of the person in the input videos. The articulation of the SMPL body also drives physically-based simulation of the clothes' Gaussians to transform the avatar to novel poses. In order to run position based dynamics simulation, physical properties including mass and material stiffness are estimated from the RGB videos through Dynamic 3D Gaussian Splatting. Experiments demonstrate that our method not only accurately reproduces appearance but also enables the reconstruction of avatars wearing highly deformable garments, such as skirts or coats, which have been challenging to reconstruct using existing methods.