GesSure- A Robust Face-Authentication enabled Dynamic Gesture Recognition GUI Application
This addresses the need for secure and natural human-machine interaction, particularly during pandemics, though it is incremental as it combines existing methods like MTCNN, FaceNet, and LSTM-CNN.
The paper tackles the problem of unreliable gesture-recognition systems lacking user authentication by designing a robust system that integrates face verification with dynamic gesture recognition, achieving 95% accuracy for five gesture classes and enabling tasks like save, print, and system controls.
Using physical interactive devices like mouse and keyboards hinders naturalistic human-machine interaction and increases the probability of surface contact during a pandemic. Existing gesture-recognition systems do not possess user authentication, making them unreliable. Static gestures in current gesture-recognition technology introduce long adaptation periods and reduce user compatibility. Our technology places a strong emphasis on user recognition and safety. We use meaningful and relevant gestures for task operation, resulting in a better user experience. This paper aims to design a robust, face-verification-enabled gesture recognition system that utilizes a graphical user interface and primarily focuses on security through user recognition and authorization. The face model uses MTCNN and FaceNet to verify the user, and our LSTM-CNN architecture for gesture recognition, achieving an accuracy of 95% with five classes of gestures. The prototype developed through our research has successfully executed context-dependent tasks like save, print, control video-player operations and exit, and context-free operating system tasks like sleep, shut-down, and unlock intuitively. Our application and dataset are available as open source.