Minimax optimal transfer learning for high-dimensional additive regression
This work addresses the problem of improving regression accuracy with limited target data by leveraging auxiliary samples, offering a theoretically grounded method for statisticians and machine learning practitioners, though it builds incrementally on existing transfer learning frameworks.
The paper tackles high-dimensional additive regression in a transfer learning setting, establishing minimax optimal error bounds for a new two-stage estimator under heavy-tailed noise conditions and demonstrating its performance through simulations and real data.
This paper studies high-dimensional additive regression under the transfer learning framework, where one observes samples from a target population together with auxiliary samples from different but potentially related regression models. We first introduce a target-only estimation procedure based on the smooth backfitting estimator with local linear smoothing. In contrast to previous work, we establish general error bounds under sub-Weibull($α$) noise, thereby accommodating heavy-tailed error distributions. In the sub-exponential case ($α=1$), we show that the estimator attains the minimax lower bound under regularity conditions, which requires a substantial departure from existing proof strategies. We then develop a novel two-stage estimation method within a transfer learning framework, and provide theoretical guarantees at both the population and empirical levels. Error bounds are derived for each stage under general tail conditions, and we further demonstrate that the minimax optimal rate is achieved when the auxiliary and target distributions are sufficiently close. All theoretical results are supported by simulation studies and real data analysis.