PV-NAS: Practical Neural Architecture Search for Video Recognition
This work provides a practical solution for automating video recognition network design, reducing the need for expert intervention and computational costs, though it appears incremental as it builds on existing neural architecture search methods.
The authors tackled the problem of automatically designing deep neural networks for video recognition, which typically requires expert knowledge and extensive trial and error. Their proposed PV-NAS method achieves state-of-the-art performance with 78.7% and 62.5% Top-1 accuracy on Kinetics-400 and Something-Something V2 datasets, outperforming previous methods by margins of 4.6% and 3.4%, respectively, while using fewer computational resources.
Recently, deep learning has been utilized to solve video recognition problem due to its prominent representation ability. Deep neural networks for video tasks is highly customized and the design of such networks requires domain experts and costly trial and error tests. Recent advance in network architecture search has boosted the image recognition performance in a large margin. However, automatic designing of video recognition network is less explored. In this study, we propose a practical solution, namely Practical Video Neural Architecture Search (PV-NAS).Our PV-NAS can efficiently search across tremendous large scale of architectures in a novel spatial-temporal network search space using the gradient based search methods. To avoid sticking into sub-optimal solutions, we propose a novel learning rate scheduler to encourage sufficient network diversity of the searched models. Extensive empirical evaluations show that the proposed PV-NAS achieves state-of-the-art performance with much fewer computational resources. 1) Within light-weight models, our PV-NAS-L achieves 78.7% and 62.5% Top-1 accuracy on Kinetics-400 and Something-Something V2, which are better than previous state-of-the-art methods (i.e., TSM) with a large margin (4.6% and 3.4% on each dataset, respectively), and 2) among median-weight models, our PV-NAS-M achieves the best performance (also a new record)in the Something-Something V2 dataset.