Image to Video Domain Adaptation Using Web Supervision
This addresses the challenge of limited labeled video data for researchers and practitioners in computer vision, though it is incremental as it builds on existing webly-supervised methods.
The paper tackles the problem of training video classifiers with scarce labeled data by leveraging noisy web-supervised data, proposing a two-stage approach that learns appearance and temporal features sequentially and incorporates domain adaptation and attention to handle noise, achieving competitive results on UCF-101 and evaluating on Kinetics.
Training deep neural networks typically requires large amounts of labeled data which may be scarce or expensive to obtain for a particular target domain. As an alternative, we can leverage webly-supervised data (i.e. results from a public search engine) which are relatively plentiful but may contain noisy results. In this work, we propose a novel two-stage approach to learn a video classifier using webly-supervised data. We argue that learning appearance features and then temporal features sequentially, rather than simultaneously, is an easier optimization for this task. We show this by first learning an image model from web images, which is used to initialize and train a video model. Our model applies domain adaptation to account for potential domain shift present between the source domain (webly-supervised data) and target domain and also accounts for noise by adding a novel attention component. We report results competitive with state-of-the-art for webly-supervised approaches on UCF-101 (while simplifying the training process) and also evaluate on Kinetics for comparison.