SRF-Net: Selective Receptive Field Network for Anchor-Free Temporal Action Detection
This work addresses the generalization limitations of anchor-based methods in video analysis for researchers and practitioners, though it appears incremental as it builds on existing one-stage detection frameworks.
The paper tackles the problem of temporal action detection in untrimmed videos by proposing an anchor-free method, SRF-Net, which directly estimates action locations and classifications without pre-defined anchors, achieving superior results on the THUMOS14 dataset compared to state-of-the-art approaches.
Temporal action detection (TAD) is a challenging task which aims to temporally localize and recognize the human action in untrimmed videos. Current mainstream one-stage TAD approaches localize and classify action proposals relying on pre-defined anchors, where the location and scale for action instances are set by designers. Obviously, such an anchor-based TAD method limits its generalization capability and will lead to performance degradation when videos contain rich action variation. In this study, we explore to remove the requirement of pre-defined anchors for TAD methods. A novel TAD model termed as Selective Receptive Field Network (SRF-Net) is developed, in which the location offsets and classification scores at each temporal location can be directly estimated in the feature map and SRF-Net is trained in an end-to-end manner. Innovatively, a building block called Selective Receptive Field Convolution (SRFC) is dedicatedly designed which is able to adaptively adjust its receptive field size according to multiple scales of input information at each temporal location in the feature map. Extensive experiments are conducted on the THUMOS14 dataset, and superior results are reported comparing to state-of-the-art TAD approaches.