An Efficient 3D Convolutional Neural Network with Channel-wise, Spatial-grouped, and Temporal Convolutions
This work addresses efficiency issues in video action recognition for researchers and practitioners, offering a novel architecture that reduces reliance on pretraining and improves performance, though it is incremental in optimizing existing 3D CNN designs.
The paper tackled the problem of inefficient video action recognition by introducing a simple and efficient 3D convolutional neural network, achieving performance that matches or surpasses larger models on datasets like Something-Something-V1&V2 and Kinetics-400, and beating previous state-of-the-art accuracy on FineGym by over 5% with only RGB input.
There has been huge progress on video action recognition in recent years. However, many works focus on tweaking existing 2D backbones due to the reliance of ImageNet pretraining, which restrains the models from achieving higher efficiency for video recognition. In this work we introduce a simple and very efficient 3D convolutional neural network for video action recognition. The design of the building block consists of a channel-wise convolution, followed by a spatial group convolution, and finally a temporal convolution. We evaluate the performance and efficiency of our proposed network on several video action recognition datasets by directly training on the target dataset without relying on pertaining. On Something-Something-V1&V2, Kinetics-400 and Multi-Moments in Time, our network can match or even surpass the performance of other models which are several times larger. On the fine-grained action recognition dataset FineGym, we beat the previous state-of-the-art accuracy achieved with 2-stream methods by more than 5% using only RGB input.