Proving the Potential of Skeleton Based Action Recognition to Automate the Analysis of Manual Processes
This work addresses the need for more flexible and cost-effective analysis of manual processes in sectors like textiles and electronics, though it is incremental as it applies existing methods to a new domain.
The paper tackled the problem of analyzing manual processes in manufacturing by using skeleton-based action recognition to detect motion classes from video streams, achieving results that demonstrate the potential of machine learning for this application with suitable generalizing approaches.
In manufacturing sectors such as textiles and electronics, manual processes are a fundamental part of production. The analysis and monitoring of the processes is necessary for efficient production design. Traditional methods for analyzing manual processes are complex, expensive, and inflexible. Compared to established approaches such as Methods-Time-Measurement (MTM), machine learning (ML) methods promise: Higher flexibility, self-sufficient & permanent use, lower costs. In this work, based on a video stream, the current motion class in a manual assembly process is detected. With information on the current motion, Key-Performance-Indicators (KPIs) can be derived easily. A skeleton-based action recognition approach is taken, as this field recently shows major success in machine vision tasks. For skeleton-based action recognition in manual assembly, no sufficient pre-work could be found. Therefore, a ML pipeline is developed, to enable extensive research on different (pre-) processing methods and neural nets. Suitable well generalizing approaches are found, proving the potential of ML to enhance analyzation of manual processes. Models detect the current motion, performed by an operator in manual assembly, but the results can be transferred to all kinds of manual processes.