Self-supervised Optimization of Hand Pose Estimation using Anatomical Features and Iterative Learning
This work addresses the problem of inaccurate hand pose estimation for manual assembly workers, but it is incremental as it builds on existing methods with filtering and retraining.
The paper tackles poor hand pose estimation in complex scenarios like wearing gloves by presenting a self-supervised pipeline that adapts hand pose estimation with minimal human interaction, enabling robust activity recognition in manual assembly.
Manual assembly workers face increasing complexity in their work. Human-centered assistance systems could help, but object recognition as an enabling technology hinders sophisticated human-centered design of these systems. At the same time, activity recognition based on hand poses suffers from poor pose estimation in complex usage scenarios, such as wearing gloves. This paper presents a self-supervised pipeline for adapting hand pose estimation to specific use cases with minimal human interaction. This enables cheap and robust hand posebased activity recognition. The pipeline consists of a general machine learning model for hand pose estimation trained on a generalized dataset, spatial and temporal filtering to account for anatomical constraints of the hand, and a retraining step to improve the model. Different parameter combinations are evaluated on a publicly available and annotated dataset. The best parameter and model combination is then applied to unlabelled videos from a manual assembly scenario. The effectiveness of the pipeline is demonstrated by training an activity recognition as a downstream task in the manual assembly scenario.