Class-Incremental Learning with Generative Classifiers
This addresses the challenge of incrementally adding new classes without storing data, offering a novel approach for machine learning systems that need to adapt over time.
The paper tackles the problem of class-incremental learning in deep neural networks by proposing a generative classification strategy that learns joint distributions p(x,y) instead of conditional ones, and it outperforms existing baselines on continual learning benchmarks.
Incrementally training deep neural networks to recognize new classes is a challenging problem. Most existing class-incremental learning methods store data or use generative replay, both of which have drawbacks, while 'rehearsal-free' alternatives such as parameter regularization or bias-correction methods do not consistently achieve high performance. Here, we put forward a new strategy for class-incremental learning: generative classification. Rather than directly learning the conditional distribution p(y|x), our proposal is to learn the joint distribution p(x,y), factorized as p(x|y)p(y), and to perform classification using Bayes' rule. As a proof-of-principle, here we implement this strategy by training a variational autoencoder for each class to be learned and by using importance sampling to estimate the likelihoods p(x|y). This simple approach performs very well on a diverse set of continual learning benchmarks, outperforming generative replay and other existing baselines that do not store data.