Counting Out Time: Class Agnostic Video Repetition Counting in the Wild
This addresses the challenge of counting repetitions in real-world videos for applications like activity analysis, though it is incremental by building on existing methods with a new dataset.
The authors tackled the problem of estimating repetition periods in videos without prior knowledge of the action class, achieving state-of-the-art performance on benchmarks like PERTUBE and QUVA with substantial improvements.
We present an approach for estimating the period with which an action is repeated in a video. The crux of the approach lies in constraining the period prediction module to use temporal self-similarity as an intermediate representation bottleneck that allows generalization to unseen repetitions in videos in the wild. We train this model, called Repnet, with a synthetic dataset that is generated from a large unlabeled video collection by sampling short clips of varying lengths and repeating them with different periods and counts. This combination of synthetic data and a powerful yet constrained model, allows us to predict periods in a class-agnostic fashion. Our model substantially exceeds the state of the art performance on existing periodicity (PERTUBE) and repetition counting (QUVA) benchmarks. We also collect a new challenging dataset called Countix (~90 times larger than existing datasets) which captures the challenges of repetition counting in real-world videos. Project webpage: https://sites.google.com/view/repnet .