When Video Classification Meets Incremental Classes
This addresses the need for video classification systems to continuously update with new classes while managing storage and computing resources, though it is an incremental advancement in incremental learning.
The paper tackles the problem of catastrophic forgetting in class-incremental video classification by proposing a framework that decomposes spatio-temporal knowledge and uses a dual granularity exemplar selection method, achieving significant performance improvements over previous state-of-the-art methods on Something-Something V2 and Kinetics datasets.
With the rapid development of social media, tremendous videos with new classes are generated daily, which raise an urgent demand for video classification methods that can continuously update new classes while maintaining the knowledge of old videos with limited storage and computing resources. In this paper, we summarize this task as Class-Incremental Video Classification (CIVC) and propose a novel framework to address it. As a subarea of incremental learning tasks, the challenge of catastrophic forgetting is unavoidable in CIVC. To better alleviate it, we utilize some characteristics of videos. First, we decompose the spatio-temporal knowledge before distillation rather than treating it as a whole in the knowledge transfer process; trajectory is also used to refine the decomposition. Second, we propose a dual granularity exemplar selection method to select and store representative video instances of old classes and key-frames inside videos under a tight storage budget. We benchmark our method and previous SOTA class-incremental learning methods on Something-Something V2 and Kinetics datasets, and our method outperforms previous methods significantly.