A General Class of Transfer Learning Regression without Implementation Cost
This work addresses the need for a general and cost-effective transfer learning method for regression tasks, though it appears incremental as it builds upon existing techniques.
The authors tackled the problem of unifying and extending transfer learning methods for regression by proposing a framework that integrates three popular approaches through density-ratio reweighting and Bayesian estimation, demonstrating its simplicity and applicability on real data without additional implementation cost.
We propose a novel framework that unifies and extends existing methods of transfer learning (TL) for regression. To bridge a pretrained source model to the model on a target task, we introduce a density-ratio reweighting function, which is estimated through the Bayesian framework with a specific prior distribution. By changing two intrinsic hyperparameters and the choice of the density-ratio model, the proposed method can integrate three popular methods of TL: TL based on cross-domain similarity regularization, a probabilistic TL using the density-ratio estimation, and fine-tuning of pretrained neural networks. Moreover, the proposed method can benefit from its simple implementation without any additional cost; the regression model can be fully trained using off-the-shelf libraries for supervised learning in which the original output variable is simply transformed to a new output variable. We demonstrate its simplicity, generality, and applicability using various real data applications.