MVTN: A Multiscale Video Transformer Network for Hand Gesture Recognition
This work addresses hand gesture recognition, a domain-specific problem, with incremental improvements in efficiency and performance.
The paper tackles dynamic hand gesture recognition by introducing a Multiscale Video Transformer Network (MVTN) that captures diverse levels of detail and context, achieving state-of-the-art results with reduced computational complexity and parameters on NVGesture and Briareo datasets.
In this paper, we introduce a novel Multiscale Video Transformer Network (MVTN) for dynamic hand gesture recognition, since multiscale features can extract features with variable size, pose, and shape of hand which is a challenge in hand gesture recognition. The proposed model incorporates a multiscale feature hierarchy to capture diverse levels of detail and context within hand gestures which enhances the model's ability. This multiscale hierarchy is obtained by extracting different dimensions of attention in different transformer stages with initial stages to model high-resolution features and later stages to model low-resolution features. Our approach also leverages multimodal data, utilizing depth maps, infrared data, and surface normals along with RGB images from NVGesture and Briareo datasets. Experiments show that the proposed MVTN achieves state-of-the-art results with less computational complexity and parameters. The source code is available at https://github.com/mallikagarg/MVTN.