Actions as Moving Points
This work addresses the need for more efficient and precise action detection in videos, which is crucial for applications like surveillance and video analysis, though it appears incremental as it builds on existing tubelet detection frameworks.
The paper tackles the problem of suboptimal and computationally expensive action tubelet detection by proposing the MovingCenter Detector (MOC-detector), which treats actions as trajectories of moving points, resulting in state-of-the-art performance on JHMDB and UCF101-24 datasets with improved precision at higher video IoU thresholds.
The existing action tubelet detectors often depend on heuristic anchor design and placement, which might be computationally expensive and sub-optimal for precise localization. In this paper, we present a conceptually simple, computationally efficient, and more precise action tubelet detection framework, termed as MovingCenter Detector (MOC-detector), by treating an action instance as a trajectory of moving points. Based on the insight that movement information could simplify and assist action tubelet detection, our MOC-detector is composed of three crucial head branches: (1) Center Branch for instance center detection and action recognition, (2) Movement Branch for movement estimation at adjacent frames to form trajectories of moving points, (3) Box Branch for spatial extent detection by directly regressing bounding box size at each estimated center. These three branches work together to generate the tubelet detection results, which could be further linked to yield video-level tubes with a matching strategy. Our MOC-detector outperforms the existing state-of-the-art methods for both metrics of frame-mAP and video-mAP on the JHMDB and UCF101-24 datasets. The performance gap is more evident for higher video IoU, demonstrating that our MOC-detector is particularly effective for more precise action detection. We provide the code at https://github.com/MCG-NJU/MOC-Detector.