Continual Learning of New Sound Classes using Generative Replay
This addresses the challenge of efficiently updating sound classifiers with new classes without retaining past data, which is incremental as it builds on existing generative replay methods.
The paper tackles the problem of catastrophic forgetting in continual learning for sound classification by developing a generative replay procedure that generates audio spectrogram data instead of storing old datasets. The result shows that using a generator 4% the size of previous training data matches the performance of keeping 20% of the data.
Continual learning consists in incrementally training a model on a sequence of datasets and testing on the union of all datasets. In this paper, we examine continual learning for the problem of sound classification, in which we wish to refine already trained models to learn new sound classes. In practice one does not want to maintain all past training data and retrain from scratch, but naively updating a model with new data(sets) results in a degradation of already learned tasks, which is referred to as "catastrophic forgetting." We develop a generative replay procedure for generating training audio spectrogram data, in place of keeping older training datasets. We show that by incrementally refining a classifier with generative replay a generator that is 4% of the size of all previous training data matches the performance of refining the classifier keeping 20% of all previous training data. We thus conclude that we can extend a trained sound classifier to learn new classes without having to keep previously used datasets.