2nd Place Scheme on Action Recognition Track of ECCV 2020 VIPriors Challenges: An Efficient Optical Flow Stream Guided Framework
This addresses the challenge of limited computational resources for organizations working with small datasets in action recognition, though it is incremental as it builds on existing two-stream methods.
The paper tackles the problem of training action recognition models on small datasets without pre-trained models, proposing a data-efficient two-stream framework that achieved 2nd place (88.31%) in the ECCV 2020 VIPriors challenge.
To address the problem of training on small datasets for action recognition tasks, most prior works are either based on a large number of training samples or require pre-trained models transferred from other large datasets to tackle overfitting problems. However, it limits the research within organizations that have strong computational abilities. In this work, we try to propose a data-efficient framework that can train the model from scratch on small datasets while achieving promising results. Specifically, by introducing a 3D central difference convolution operation, we proposed a novel C3D neural network-based two-stream (Rank Pooling RGB and Optical Flow) framework for the task. The method is validated on the action recognition track of the ECCV 2020 VIPriors challenges and got the 2nd place (88.31%). It is proved that our method can achieve a promising result even without a pre-trained model on large scale datasets. The code will be released soon.