Live and Learn: Continual Action Clustering with Incremental Views
This addresses a practical limitation in multi-view action clustering for scenarios where camera views are not all available upfront, offering an incremental solution.
The paper tackles the problem of multi-view action clustering when camera views are added incrementally over time, proposing a continual action clustering method that achieves excellent clustering performance and time/space efficiency compared to 15 state-of-the-art baselines on 6 realistic datasets.
Multi-view action clustering leverages the complementary information from different camera views to enhance the clustering performance. Although existing approaches have achieved significant progress, they assume all camera views are available in advance, which is impractical when the camera view is incremental over time. Besides, learning the invariant information among multiple camera views is still a challenging issue, especially in continual learning scenario. Aiming at these problems, we propose a novel continual action clustering (CAC) method, which is capable of learning action categories in a continual learning manner. To be specific, we first devise a category memory library, which captures and stores the learned categories from historical views. Then, as a new camera view arrives, we only need to maintain a consensus partition matrix, which can be updated by leveraging the incoming new camera view rather than keeping all of them. Finally, a three-step alternate optimization is proposed, in which the category memory library and consensus partition matrix are optimized. The empirical experimental results on 6 realistic multi-view action collections demonstrate the excellent clustering performance and time/space efficiency of the CAC compared with 15 state-of-the-art baselines.