Robust and flexible learning of a high-dimensional classification rule using auxiliary outcomes
This work addresses bias in high-dimensional classification for researchers and practitioners using correlated outcomes, offering an incremental improvement over traditional multi-task learning methods.
The paper tackles the problem of biased estimation in multi-task learning when auxiliary outcomes are present, proposing a robust transfer learning approach that combines multi-task learning with calibration to reduce estimation error for the target outcome. Simulations and real data analysis demonstrate the method's superiority over single-outcome approaches.
Correlated outcomes are common in many practical problems. In some settings, one outcome is of particular interest, and others are auxiliary. To leverage information shared by all the outcomes, traditional multi-task learning (MTL) minimizes an averaged loss function over all the outcomes, which may lead to biased estimation for the target outcome, especially when the MTL model is mis-specified. In this work, based on a decomposition of estimation bias into two types, within-subspace and against-subspace, we develop a robust transfer learning approach to estimating a high-dimensional linear decision rule for the outcome of interest with the presence of auxiliary outcomes. The proposed method includes an MTL step using all outcomes to gain efficiency, and a subsequent calibration step using only the outcome of interest to correct both types of biases. We show that the final estimator can achieve a lower estimation error than the one using only the single outcome of interest. Simulations and real data analysis are conducted to justify the superiority of the proposed method.