Temporal Action Localization using Long Short-Term Dependency
This work addresses a key challenge in video analysis for applications like surveillance and content indexing, representing an incremental improvement over existing methods.
The paper tackles temporal action localization in untrimmed videos by introducing the Gemini Network, which models temporal structures using two subnets, parallel feature extraction, and auxiliary supervision, achieving state-of-the-art performance on THUMOS14 and ActivityNet datasets.
Temporal action localization in untrimmed videos is an important but difficult task. Difficulties are encountered in the application of existing methods when modeling temporal structures of videos. In the present study, we developed a novel method, referred to as Gemini Network, for effective modeling of temporal structures and achieving high-performance temporal action localization. The significant improvements afforded by the proposed method are attributable to three major factors. First, the developed network utilizes two subnets for effective modeling of temporal structures. Second, three parallel feature extraction pipelines are used to prevent interference between the extractions of different stage features. Third, the proposed method utilizes auxiliary supervision, with the auxiliary classifier losses affording additional constraints for improving the modeling capability of the network. As a demonstration of its effectiveness, the Gemini Network was used to achieve state-of-the-art temporal action localization performance on two challenging datasets, namely, THUMOS14 and ActivityNet.