Egocentric Visibility-Aware Human Pose Estimation
This addresses a key bottleneck for VR/AR applications by improving pose estimation accuracy when body parts are occluded from the head-mounted camera view.
The paper tackles the problem of egocentric human pose estimation where keypoints are often invisible, by introducing Eva-3M, a large-scale dataset with over 3.0M frames including 435K with visibility annotations, and EvaPose, a method that explicitly uses visibility information. The result is state-of-the-art performance on Eva-3M and EMHI datasets, demonstrating the value of ground-truth visibility labels.
Egocentric human pose estimation (HPE) using a head-mounted device is crucial for various VR and AR applications, but it faces significant challenges due to keypoint invisibility. Nevertheless, none of the existing egocentric HPE datasets provide keypoint visibility annotations, and the existing methods often overlook the invisibility problem, treating visible and invisible keypoints indiscriminately during estimation. As a result, their capacity to accurately predict visible keypoints is compromised. In this paper, we first present Eva-3M, a large-scale egocentric visibility-aware HPE dataset comprising over 3.0M frames, with 435K of them annotated with keypoint visibility labels. Additionally, we augment the existing EMHI dataset with keypoint visibility annotations to further facilitate the research in this direction. Furthermore, we propose EvaPose, a novel egocentric visibility-aware HPE method that explicitly incorporates visibility information to enhance pose estimation accuracy. Extensive experiments validate the significant value of ground-truth visibility labels in egocentric HPE settings, and demonstrate that our EvaPose achieves state-of-the-art performance in both Eva-3M and EMHI datasets.