Large-scale Isolated Gesture Recognition Using Convolutional Neural Networks
This work addresses gesture recognition for human-computer interaction, offering an incremental improvement by adapting existing ConvNets to depth sequences with new representations.
The paper tackled large-scale isolated gesture recognition by proposing three compact depth sequence representations (DDI, DDNI, DDMNI) using bidirectional rank pooling, enabling fine-tuning of existing ConvNets without adding many parameters. It achieved 55.57% accuracy and ranked 2nd in the ChaLearn LAP 2016 challenge, using only depth data and performing close to the best.
This paper proposes three simple, compact yet effective representations of depth sequences, referred to respectively as Dynamic Depth Images (DDI), Dynamic Depth Normal Images (DDNI) and Dynamic Depth Motion Normal Images (DDMNI). These dynamic images are constructed from a sequence of depth maps using bidirectional rank pooling to effectively capture the spatial-temporal information. Such image-based representations enable us to fine-tune the existing ConvNets models trained on image data for classification of depth sequences, without introducing large parameters to learn. Upon the proposed representations, a convolutional Neural networks (ConvNets) based method is developed for gesture recognition and evaluated on the Large-scale Isolated Gesture Recognition at the ChaLearn Looking at People (LAP) challenge 2016. The method achieved 55.57\% classification accuracy and ranked $2^{nd}$ place in this challenge but was very close to the best performance even though we only used depth data.