ConvNet Architecture Search for Spatiotemporal Feature Learning
This work addresses the need for efficient and effective video representation models for computer vision tasks, though it is incremental as it builds on existing ConvNet and architecture search methods.
The paper tackled the problem of learning spatiotemporal features for video understanding by conducting an empirical ConvNet architecture search, resulting in a deep 3D Residual ConvNet that outperforms C3D on multiple benchmarks while being faster, smaller, and more compact.
Learning image representations with ConvNets by pre-training on ImageNet has proven useful across many visual understanding tasks including object detection, semantic segmentation, and image captioning. Although any image representation can be applied to video frames, a dedicated spatiotemporal representation is still vital in order to incorporate motion patterns that cannot be captured by appearance based models alone. This paper presents an empirical ConvNet architecture search for spatiotemporal feature learning, culminating in a deep 3-dimensional (3D) Residual ConvNet. Our proposed architecture outperforms C3D by a good margin on Sports-1M, UCF101, HMDB51, THUMOS14, and ASLAN while being 2 times faster at inference time, 2 times smaller in model size, and having a more compact representation.