ETCH: Generalizing Body Fitting to Clothed Humans via Equivariant Tightness
This addresses a common yet difficult problem in computer vision and graphics for applications like animation and virtual try-on, offering a novel approach that generalizes better than existing methods.
The paper tackles the problem of fitting a body to 3D clothed human point clouds, which is challenging due to pose and garment diversity, and proposes ETCH, a pipeline that uses equivariant tightness mapping to simplify the task, resulting in significant accuracy improvements such as 16.7% to 69.5% on loose clothing and 49.9% average shape accuracy.
Fitting a body to a 3D clothed human point cloud is a common yet challenging task. Traditional optimization-based approaches use multi-stage pipelines that are sensitive to pose initialization, while recent learning-based methods often struggle with generalization across diverse poses and garment types. We propose Equivariant Tightness Fitting for Clothed Humans, or ETCH, a novel pipeline that estimates cloth-to-body surface mapping through locally approximate SE(3) equivariance, encoding tightness as displacement vectors from the cloth surface to the underlying body. Following this mapping, pose-invariant body features regress sparse body markers, simplifying clothed human fitting into an inner-body marker fitting task. Extensive experiments on CAPE and 4D-Dress show that ETCH significantly outperforms state-of-the-art methods -- both tightness-agnostic and tightness-aware -- in body fitting accuracy on loose clothing (16.7% ~ 69.5%) and shape accuracy (average 49.9%). Our equivariant tightness design can even reduce directional errors by (67.2% ~ 89.8%) in one-shot (or out-of-distribution) settings (~ 1% data). Qualitative results demonstrate strong generalization of ETCH, regardless of challenging poses, unseen shapes, loose clothing, and non-rigid dynamics. We will release the code and models soon for research purposes at https://boqian-li.github.io/ETCH/.