Egocentric View Hand Action Recognition by Leveraging Hand Surface and Hand Grasp Type
This work addresses hand action recognition for applications like robotics or AR/VR, but it is incremental as it builds on existing methods without a major breakthrough.
The paper tackles hand action recognition in egocentric videos by using mean curvature of hand surface and hand grasp type, achieving improved performance as noted in experiments.
We introduce a multi-stage framework that uses mean curvature on a hand surface and focuses on learning interaction between hand and object by analyzing hand grasp type for hand action recognition in egocentric videos. The proposed method does not require 3D information of objects including 6D object poses which are difficult to annotate for learning an object's behavior while it interacts with hands. Instead, the framework synthesizes the mean curvature of the hand mesh model to encode the hand surface geometry in 3D space. Additionally, our method learns the hand grasp type which is highly correlated with the hand action. From our experiment, we notice that using hand grasp type and mean curvature of hand increases the performance of the hand action recognition.