Challenges of the Creation of a Dataset for Vision Based Human Hand Action Recognition in Industrial Assembly
This provides a specialized dataset for vision-based hand action recognition in industrial assembly, addressing a domain-specific need with legal and technical compliance.
The authors created the Industrial Hand Action Dataset V1 containing 459,180-2,295,900 images of 12 industrial assembly hand action classes, addressing gaps in existing datasets, and demonstrated its usability by achieving 86.25% test accuracy with an adapted Gated Transformer Network.
This work presents the Industrial Hand Action Dataset V1, an industrial assembly dataset consisting of 12 classes with 459,180 images in the basic version and 2,295,900 images after spatial augmentation. Compared to other freely available datasets tested, it has an above-average duration and, in addition, meets the technical and legal requirements for industrial assembly lines. Furthermore, the dataset contains occlusions, hand-object interaction, and various fine-grained human hand actions for industrial assembly tasks that were not found in combination in examined datasets. The recorded ground truth assembly classes were selected after extensive observation of real-world use cases. A Gated Transformer Network, a state-of-the-art model from the transformer domain was adapted, and proved with a test accuracy of 86.25% before hyperparameter tuning by 18,269,959 trainable parameters, that it is possible to train sequential deep learning models with this dataset.