Near-Optimal Linear Regression under Distribution Shift
This addresses the problem of transfer learning for researchers and practitioners needing robust regression methods when data distributions shift, though it appears incremental as it builds on existing minimax frameworks.
The paper tackles linear regression under distribution shift by developing estimators that achieve minimax linear risk, covering settings like covariate and model shift, and shows these estimators are within a constant factor of the minimax risk even among nonlinear estimators.
Transfer learning is essential when sufficient data comes from the source domain, with scarce labeled data from the target domain. We develop estimators that achieve minimax linear risk for linear regression problems under distribution shift. Our algorithms cover different transfer learning settings including covariate shift and model shift. We also consider when data are generated from either linear or general nonlinear models. We show that linear minimax estimators are within an absolute constant of the minimax risk even among nonlinear estimators for various source/target distributions.