Hand Guided High Resolution Feature Enhancement for Fine-Grained Atomic Action Segmentation within Complex Human Assemblies
This work addresses the need for accurate, real-time action segmentation in manufacturing for human-robot collaboration, though it appears incremental as it builds on existing encoder/decoder methods with specific enhancements.
The paper tackles the problem of fine-grained atomic action segmentation in complex human assembly videos, where traditional methods lose vital spatial and temporal information due to downsampling. It presents a hand location guided high-resolution feature enhancement model that, combined with surround sampling during training and temporally aware label cleaning at inference, achieves real-time classification and surpasses similar encoder/decoder methods on a novel dataset of 24 atomic actions from a robotics assembly production line.
Due to the rapid temporal and fine-grained nature of complex human assembly atomic actions, traditional action segmentation approaches requiring the spatial (and often temporal) down sampling of video frames often loose vital fine-grained spatial and temporal information required for accurate classification within the manufacturing domain. In order to fully utilise higher resolution video data (often collected within the manufacturing domain) and facilitate real time accurate action segmentation - required for human robot collaboration - we present a novel hand location guided high resolution feature enhanced model. We also propose a simple yet effective method of deploying offline trained action recognition models for real time action segmentation on temporally short fine-grained actions, through the use of surround sampling while training and temporally aware label cleaning at inference. We evaluate our model on a novel action segmentation dataset containing 24 (+background) atomic actions from video data of a real world robotics assembly production line. Showing both high resolution hand features as well as traditional frame wide features improve fine-grained atomic action classification, and that though temporally aware label clearing our model is capable of surpassing similar encoder/decoder methods, while allowing for real time classification.