3D Human Keypoints Estimation From Point Clouds in the Wild Without Human Labels
This addresses the costly and error-prone labeling issue for 3D human keypoints, especially for rare cases like pedestrians with unusual poses, offering an incremental improvement in unsupervised learning for computer vision.
The paper tackles the problem of 3D human keypoint estimation from point clouds without human labels by proposing GC-KPL, which uses unsupervised geometry consistency losses, achieving performance comparable to fully supervised methods and enabling few-shot learning with only 10% labeled data.
Training a 3D human keypoint detector from point clouds in a supervised manner requires large volumes of high quality labels. While it is relatively easy to capture large amounts of human point clouds, annotating 3D keypoints is expensive, subjective, error prone and especially difficult for long-tail cases (pedestrians with rare poses, scooterists, etc.). In this work, we propose GC-KPL - Geometry Consistency inspired Key Point Leaning, an approach for learning 3D human joint locations from point clouds without human labels. We achieve this by our novel unsupervised loss formulations that account for the structure and movement of the human body. We show that by training on a large training set from Waymo Open Dataset without any human annotated keypoints, we are able to achieve reasonable performance as compared to the fully supervised approach. Further, the backbone benefits from the unsupervised training and is useful in downstream fewshot learning of keypoints, where fine-tuning on only 10 percent of the labeled training data gives comparable performance to fine-tuning on the entire set. We demonstrated that GC-KPL outperforms by a large margin over SoTA when trained on entire dataset and efficiently leverages large volumes of unlabeled data.