A Unified Method for First and Third Person Action Recognition
This addresses the problem of unified action recognition for video analysis applications, but it is incremental as it builds on existing two-stream approaches.
The paper tackles action recognition in both first- and third-person videos by proposing a two-stream method that captures complementary appearance and motion features, achieving state-of-the-art results on relevant datasets.
In this paper, a new video classification methodology is proposed which can be applied in both first and third person videos. The main idea behind the proposed strategy is to capture complementary information of appearance and motion efficiently by performing two independent streams on the videos. The first stream is aimed to capture long-term motions from shorter ones by keeping track of how elements in optical flow images have changed over time. Optical flow images are described by pre-trained networks that have been trained on large scale image datasets. A set of multi-channel time series are obtained by aligning descriptions beside each other. For extracting motion features from these time series, PoT representation method plus a novel pooling operator is followed due to several advantages. The second stream is accomplished to extract appearance features which are vital in the case of video classification. The proposed method has been evaluated on both first and third-person datasets and results present that the proposed methodology reaches the state of the art successfully.