Overcoming Concept Shift in Domain-Aware Settings through Consolidated Internal Distributions
This addresses domain adaptation challenges for machine learning practitioners when source data is inaccessible, but it appears incremental as it builds on existing adaptation methods.
The paper tackles the problem of concept shift in domain-aware settings by developing an algorithm that improves pre-trained model performance without full retraining, using only unannotated samples of initial concepts, and demonstrates effectiveness through experiments.
We develop an algorithm to improve the performance of a pre-trained model under concept shift without retraining the model from scratch when only unannotated samples of initial concepts are accessible. We model this problem as a domain adaptation problem, where the source domain data is inaccessible during model adaptation. The core idea is based on consolidating the intermediate internal distribution, learned to represent the source domain data, after adapting the model. We provide theoretical analysis and conduct extensive experiments to demonstrate that the proposed method is effective.