Mining YouTube - A dataset for learning fine-grained action concepts from webly supervised video data
This addresses the need for less restricted scenarios in action recognition for researchers, but it is incremental as it builds on existing webly supervised approaches.
The authors tackled the problem of action recognition by moving beyond fully supervised classification on hand-crafted datasets, creating a large-scale real-world dataset from YouTube videos with 250 cooking videos for testing and training data mined from subtitles without human intervention, and proposed a semantic hierarchical structure to address inconsistencies.
Action recognition is so far mainly focusing on the problem of classification of hand selected preclipped actions and reaching impressive results in this field. But with the performance even ceiling on current datasets, it also appears that the next steps in the field will have to go beyond this fully supervised classification. One way to overcome those problems is to move towards less restricted scenarios. In this context we present a large-scale real-world dataset designed to evaluate learning techniques for human action recognition beyond hand-crafted datasets. To this end we put the process of collecting data on its feet again and start with the annotation of a test set of 250 cooking videos. The training data is then gathered by searching for the respective annotated classes within the subtitles of freely available videos. The uniqueness of the dataset is attributed to the fact that the whole process of collecting the data and training does not involve any human intervention. To address the problem of semantic inconsistencies that arise with this kind of training data, we further propose a semantical hierarchical structure for the mined classes.