An Animation-based Augmentation Approach for Action Recognition from Discontinuous Video
This addresses a frequent real-world scenario in computer vision for applications like surveillance or human-computer interaction, but it is incremental as it builds on existing pose estimation and augmentation techniques.
The paper tackles the problem of action recognition performance decline when trained with discontinuous video frames by introducing the 4A pipeline, which generates realistic animations to bridge the domain gap between virtual and real-world data, achieving comparable or superior performance with only 10% of the original data volume.
Action recognition, an essential component of computer vision, plays a pivotal role in multiple applications. Despite significant improvements brought by Convolutional Neural Networks (CNNs), these models suffer performance declines when trained with discontinuous video frames, which is a frequent scenario in real-world settings. This decline primarily results from the loss of temporal continuity, which is crucial for understanding the semantics of human actions. To overcome this issue, we introduce the 4A (Action Animation-based Augmentation Approach) pipeline, which employs a series of sophisticated techniques: starting with 2D human pose estimation from RGB videos, followed by Quaternion-based Graph Convolution Network for joint orientation and trajectory prediction, and Dynamic Skeletal Interpolation for creating smoother, diversified actions using game engine technology. This innovative approach generates realistic animations in varied game environments, viewed from multiple viewpoints. In this way, our method effectively bridges the domain gap between virtual and real-world data. In experimental evaluations, the 4A pipeline achieves comparable or even superior performance to traditional training approaches using real-world data, while requiring only 10% of the original data volume. Additionally, our approach demonstrates enhanced performance on In-the-wild videos, marking a significant advancement in the field of action recognition.