Continual Gesture Learning without Data via Synthetic Feature Sampling
This work addresses continual learning for gesture recognition in VR/AR applications, where data for old classes is unavailable, representing an incremental advancement in a domain-specific context.
The paper tackled the problem of data-free class incremental learning for skeleton-based gesture classification by developing Synthetic Feature Replay, which samples synthetic features from prototypes to replay old classes and augment new ones, achieving up to 15% improvements in mean accuracy and reducing accuracy imbalance.
Data-Free Class Incremental Learning (DFCIL) aims to enable models to continuously learn new classes while retraining knowledge of old classes, even when the training data for old classes is unavailable. Although explored primarily with image datasets by researchers, this study focuses on investigating DFCIL for skeleton-based gesture classification due to its significant real-world implications, particularly considering the growing prevalence of VR/AR headsets where gestures serve as the primary means of control and interaction. In this work, we made an intriguing observation: skeleton models trained with base classes(even very limited) demonstrate strong generalization capabilities to unseen classes without requiring additional training. Building on this insight, we developed Synthetic Feature Replay (SFR) that can sample synthetic features from class prototypes to replay for old classes and augment for new classes (under a few-shot setting). Our proposed method showcases significant advancements over the state-of-the-art, achieving up to 15% enhancements in mean accuracy across all steps and largely mitigating the accuracy imbalance between base classes and new classes.