Adversarial Learning Networks: Source-free Unsupervised Domain Incremental Learning
This work addresses the challenge of adapting DNNs to changing data distributions for applications like sentiment and disease prediction, though it appears incremental as it builds on existing continual learning and domain adaptation techniques.
The paper tackles the problem of updating deep neural network models in non-stationary environments without storing past data, using an unsupervised source-free method with Gaussian prototypes and domain adaptation, achieving improved performance and minimal forgetting in incremental sentiment and disease prediction tasks.
This work presents an approach for incrementally updating deep neural network (DNN) models in a non-stationary environment. DNN models are sensitive to changes in input data distribution, which limits their application to problem settings with stationary input datasets. In a non-stationary environment, updating a DNN model requires parameter re-training or model fine-tuning. We propose an unsupervised source-free method to update DNN classification models. The contributions of this work are two-fold. First, we use trainable Gaussian prototypes to generate representative samples for future iterations; second, using unsupervised domain adaptation, we incrementally adapt the existing model using unlabelled data. Unlike existing methods, our approach can update a DNN model incrementally for non-stationary source and target tasks without storing past training data. We evaluated our work on incremental sentiment prediction and incremental disease prediction applications and compared our approach to state-of-the-art continual learning, domain adaptation, and ensemble learning methods. Our results show that our approach achieved improved performance compared to existing incremental learning methods. We observe minimal forgetting of past knowledge over many iterations, which can help us develop unsupervised self-learning systems.