Adapting to Continuous Covariate Shift via Online Density Ratio Estimation
This addresses a challenging scenario in machine learning for applications with streaming data, though it is incremental as it builds on importance-weighted learning for covariate shift.
The paper tackles the problem of continuous covariate shift, where test data distributions shift sequentially over time, by proposing an online density ratio estimation method to adaptively train a predictor and minimize accumulated prediction risk, achieving theoretical guarantees and empirical validation.
Dealing with distribution shifts is one of the central challenges for modern machine learning. One fundamental situation is the covariate shift, where the input distributions of data change from training to testing stages while the input-conditional output distribution remains unchanged. In this paper, we initiate the study of a more challenging scenario -- continuous covariate shift -- in which the test data appear sequentially, and their distributions can shift continuously. Our goal is to adaptively train the predictor such that its prediction risk accumulated over time can be minimized. Starting with the importance-weighted learning, we show the method works effectively if the time-varying density ratios of test and train inputs can be accurately estimated. However, existing density ratio estimation methods would fail due to data scarcity at each time step. To this end, we propose an online method that can appropriately reuse historical information. Our density ratio estimation method is proven to perform well by enjoying a dynamic regret bound, which finally leads to an excess risk guarantee for the predictor. Empirical results also validate the effectiveness.