Recognizing Actions in Videos from Unseen Viewpoints
This addresses the limitation of video action recognition models for applications where training data cannot cover all possible viewpoints, though it is incremental as it builds on existing CNN methods.
The paper tackles the problem of recognizing actions in videos from camera viewpoints not present in training data, showing that standard CNN models fail in this scenario, and introduces a new geometric convolutional layer and 3D representations to achieve viewpoint invariance, with results demonstrated on a new challenging dataset.
Standard methods for video recognition use large CNNs designed to capture spatio-temporal data. However, training these models requires a large amount of labeled training data, containing a wide variety of actions, scenes, settings and camera viewpoints. In this paper, we show that current convolutional neural network models are unable to recognize actions from camera viewpoints not present in their training data (i.e., unseen view action recognition). To address this, we develop approaches based on 3D representations and introduce a new geometric convolutional layer that can learn viewpoint invariant representations. Further, we introduce a new, challenging dataset for unseen view recognition and show the approaches ability to learn viewpoint invariant representations.