Two-Stream Consensus Network for Weakly-Supervised Temporal Action Localization
This addresses the challenge of localizing actions in videos without frame-level annotations, which is incremental as it builds on existing weakly-supervised methods.
The paper tackles the problem of weakly-supervised temporal action localization by proposing a Two-Stream Consensus Network (TSCN) to improve action proposal accuracy and reduce false positives, achieving state-of-the-art results on THUMOS14 and ActivityNet datasets and comparable performance to some fully-supervised methods.
Weakly-supervised Temporal Action Localization (W-TAL) aims to classify and localize all action instances in an untrimmed video under only video-level supervision. However, without frame-level annotations, it is challenging for W-TAL methods to identify false positive action proposals and generate action proposals with precise temporal boundaries. In this paper, we present a Two-Stream Consensus Network (TSCN) to simultaneously address these challenges. The proposed TSCN features an iterative refinement training method, where a frame-level pseudo ground truth is iteratively updated, and used to provide frame-level supervision for improved model training and false positive action proposal elimination. Furthermore, we propose a new attention normalization loss to encourage the predicted attention to act like a binary selection, and promote the precise localization of action instance boundaries. Experiments conducted on the THUMOS14 and ActivityNet datasets show that the proposed TSCN outperforms current state-of-the-art methods, and even achieves comparable results with some recent fully-supervised methods.