Quo Vadis, Skeleton Action Recognition ?
This work addresses the problem of evaluating action recognition models in diverse, uncontrolled environments for researchers, though it is incremental as it focuses on dataset creation and benchmarking rather than novel methods.
The paper tackles the challenge of skeleton-based human action recognition in real-world scenarios by introducing three new datasets (Skeletics-152, Skeleton-Mimetics, and Metaphorics) to benchmark existing models, revealing domain gaps and limitations in current approaches.
In this paper, we study current and upcoming frontiers across the landscape of skeleton-based human action recognition. To study skeleton-action recognition in the wild, we introduce Skeletics-152, a curated and 3-D pose-annotated subset of RGB videos sourced from Kinetics-700, a large-scale action dataset. We extend our study to include out-of-context actions by introducing Skeleton-Mimetics, a dataset derived from the recently introduced Mimetics dataset. We also introduce Metaphorics, a dataset with caption-style annotated YouTube videos of the popular social game Dumb Charades and interpretative dance performances. We benchmark state-of-the-art models on the NTU-120 dataset and provide multi-layered assessment of the results. The results from benchmarking the top performers of NTU-120 on the newly introduced datasets reveal the challenges and domain gap induced by actions in the wild. Overall, our work characterizes the strengths and limitations of existing approaches and datasets. Via the introduced datasets, our work enables new frontiers for human action recognition.