Automatic Interaction and Activity Recognition from Videos of Human Manual Demonstrations with Application to Anomaly Detection
This work addresses activity recognition for applications like anomaly detection, but it appears incremental as it builds on existing Scene Graph methods.
The paper tackled the problem of recognizing interactions and activities from videos of manual tasks by describing spatio-temporal relations using Scene Graphs, and it demonstrated effectiveness in experiments with multi-subject data, showing recognition and clustering without prior knowledge.
This paper presents a new method to describe spatio-temporal relations between objects and hands, to recognize both interactions and activities within video demonstrations of manual tasks. The approach exploits Scene Graphs to extract key interaction features from image sequences while simultaneously encoding motion patterns and context. Additionally, the method introduces event-based automatic video segmentation and clustering, which allow for the grouping of similar events and detect if a monitored activity is executed correctly. The effectiveness of the approach was demonstrated in two multi-subject experiments, showing the ability to recognize and cluster hand-object and object-object interactions without prior knowledge of the activity, as well as matching the same activity performed by different subjects.