ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search
This work addresses the need for efficient, real-time video pose estimation, which is incremental as it builds on existing NAS methods by extending search to temporal features.
The paper tackles the problem of balancing accuracy and efficiency in video pose estimation by proposing ViPNAS, a neural architecture search method that searches networks at spatial and temporal levels, achieving real-time inference speed on CPU without sacrificing accuracy on COCO2017 and PoseTrack2018 datasets.
Human pose estimation has achieved significant progress in recent years. However, most of the recent methods focus on improving accuracy using complicated models and ignoring real-time efficiency. To achieve a better trade-off between accuracy and efficiency, we propose a novel neural architecture search (NAS) method, termed ViPNAS, to search networks in both spatial and temporal levels for fast online video pose estimation. In the spatial level, we carefully design the search space with five different dimensions including network depth, width, kernel size, group number, and attentions. In the temporal level, we search from a series of temporal feature fusions to optimize the total accuracy and speed across multiple video frames. To the best of our knowledge, we are the first to search for the temporal feature fusion and automatic computation allocation in videos. Extensive experiments demonstrate the effectiveness of our approach on the challenging COCO2017 and PoseTrack2018 datasets. Our discovered model family, S-ViPNAS and T-ViPNAS, achieve significantly higher inference speed (CPU real-time) without sacrificing the accuracy compared to the previous state-of-the-art methods.