Developing Motion Code Embedding for Action Recognition in Videos
This work offers an incremental improvement in action recognition accuracy for computer vision researchers working with egocentric video datasets.
This paper proposes 'motion codes,' a vectorized representation of motion based on mechanical attributes, obtained through a motion taxonomy. Integrating these motion codes into a state-of-the-art action recognition model resulted in higher accuracy for verb classification on egocentric videos from the EPIC-KITCHENS dataset compared to the baseline.
In this work, we propose a motion embedding strategy known as motion codes, which is a vectorized representation of motions based on a manipulation's salient mechanical attributes. These motion codes provide a robust motion representation, and they are obtained using a hierarchy of features called the motion taxonomy. We developed and trained a deep neural network model that combines visual and semantic features to identify the features found in our motion taxonomy to embed or annotate videos with motion codes. To demonstrate the potential of motion codes as features for machine learning tasks, we integrated the extracted features from the motion embedding model into the current state-of-the-art action recognition model. The obtained model achieved higher accuracy than the baseline model for the verb classification task on egocentric videos from the EPIC-KITCHENS dataset.