Tackling Online One-Class Incremental Learning by Removing Negative Contrasts
This work addresses a specific bottleneck in continual learning for scenarios where new classes arrive one at a time, offering an incremental improvement over existing methods.
The paper tackles the problem of online one-class incremental learning, where a model must learn from a stream of data with new classes introduced one at a time, by adapting a self-supervised learning approach (BYOL) to supervised learning and adding regularization on class prototypes. The result is a new method that achieves strong performance in the one-class setting and is competitive with top methods in multi-class incremental learning.
Recent work studies the supervised online continual learning setting where a learner receives a stream of data whose class distribution changes over time. Distinct from other continual learning settings the learner is presented new samples only once and must distinguish between all seen classes. A number of successful methods in this setting focus on storing and replaying a subset of samples alongside incoming data in a computationally efficient manner. One recent proposal ER-AML achieved strong performance in this setting by applying an asymmetric loss based on contrastive learning to the incoming data and replayed data. However, a key ingredient of the proposed method is avoiding contrasts between incoming data and stored data, which makes it impractical for the setting where only one new class is introduced in each phase of the stream. In this work we adapt a recently proposed approach (\textit{BYOL}) from self-supervised learning to the supervised learning setting, unlocking the constraint on contrasts. We then show that supplementing this with additional regularization on class prototypes yields a new method that achieves strong performance in the one-class incremental learning setting and is competitive with the top performing methods in the multi-class incremental setting.