On the Use of Sparse Filtering for Covariate Shift Adaptation
This addresses covariate shift adaptation for machine learning practitioners, offering a more flexible method, though it is incremental as it builds on existing sparse filtering techniques.
The paper tackles covariate shift adaptation by analyzing sparse filtering, proving it works only under a restrictive cosine metric condition, and proposes periodic sparse filtering that works under a looser periodic structure condition, achieving competitive performance on real-world datasets.
In this paper we formally analyse the use of sparse filtering algorithms to perform covariate shift adaptation. We provide a theoretical analysis of sparse filtering by evaluating the conditions required to perform covariate shift adaptation. We prove that sparse filtering can perform adaptation only if the conditional distribution of the labels has a structure explained by a cosine metric. To overcome this limitation, we propose a new algorithm, named periodic sparse filtering, and carry out the same theoretical analysis regarding covariate shift adaptation. We show that periodic sparse filtering can perform adaptation under the looser and more realistic requirement that the conditional distribution of the labels has a periodic structure, which may be satisfied, for instance, by user-dependent data sets. We experimentally validate our theoretical results on synthetic data. Moreover, we apply periodic sparse filtering to real-world data sets to demonstrate that this simple and computationally efficient algorithm is able to achieve competitive performances.