Leveraging 2D Masked Reconstruction for Domain Adaptation of 3D Pose Estimation
This addresses the challenge of domain shift in 3D pose estimation for computer vision applications, but it is incremental as it builds on existing masked image modeling techniques.
The paper tackles the problem of 3D pose estimation failing on images with distributions different from training data by introducing an unsupervised domain adaptation framework using masked image modeling, achieving state-of-the-art accuracy on various datasets.
RGB-based 3D pose estimation methods have been successful with the development of deep learning and the emergence of high-quality 3D pose datasets. However, most existing methods do not operate well for testing images whose distribution is far from that of training data. However, most existing methods do not operate well for testing images whose distribution is far from that of training data. This problem might be alleviated by involving diverse data during training, however it is non-trivial to collect such diverse data with corresponding labels (i.e. 3D pose). In this paper, we introduced an unsupervised domain adaptation framework for 3D pose estimation that utilizes the unlabeled data in addition to labeled data via masked image modeling (MIM) framework. Foreground-centric reconstruction and attention regularization are further proposed to increase the effectiveness of unlabeled data usage. Experiments are conducted on the various datasets in human and hand pose estimation tasks, especially using the cross-domain scenario. We demonstrated the effectiveness of ours by achieving the state-of-the-art accuracy on all datasets.