Faster and Accurate Compressed Video Action Recognition Straight from the Frequency Domain
This work provides a more efficient approach to video action recognition for applications where computational load and memory usage are critical, such as surveillance and smart homes.
This paper addresses the computational cost of decoding compressed video for action recognition by proposing a deep neural network that directly processes compressed video data. The method achieves comparable recognition performance to state-of-the-art methods on UCF-101 and HMDB-51 datasets, while running up to 2 times faster in inference speed.
Human action recognition has become one of the most active field of research in computer vision due to its wide range of applications, like surveillance, medical, industrial environments, smart homes, among others. Recently, deep learning has been successfully used to learn powerful and interpretable features for recognizing human actions in videos. Most of the existing deep learning approaches have been designed for processing video information as RGB image sequences. For this reason, a preliminary decoding process is required, since video data are often stored in a compressed format. However, a high computational load and memory usage is demanded for decoding a video. To overcome this problem, we propose a deep neural network capable of learning straight from compressed video. Our approach was evaluated on two public benchmarks, the UCF-101 and HMDB-51 datasets, demonstrating comparable recognition performance to the state-of-the-art methods, with the advantage of running up to 2 times faster in terms of inference speed.