To transfer or not transfer: Unified transferability metric and analysis
This work addresses a fundamental gap in transfer learning by providing a practical decision-making tool for researchers and practitioners, though it is incremental as it builds on existing methods to handle broader scenarios.
The authors tackled the problem of evaluating transferability in transfer learning by proposing a unified metric and analytical approach, WDJE, which effectively determines whether to transfer or not in classification and regression tasks with domain and task differences, as demonstrated in experiments on image classification and remaining useful life regression prediction.
In transfer learning, transferability is one of the most fundamental problems, which aims to evaluate the effectiveness of arbitrary transfer tasks. Existing research focuses on classification tasks and neglects domain or task differences. More importantly, there is a lack of research to determine whether to transfer or not. To address these, we propose a new analytical approach and metric, Wasserstein Distance based Joint Estimation (WDJE), for transferability estimation and determination in a unified setting: classification and regression problems with domain and task differences. The WDJE facilitates decision-making on whether to transfer or not by comparing the target risk with and without transfer. To enable the comparison, we approximate the target transfer risk by proposing a non-symmetric, easy-to-understand and easy-to-calculate target risk bound that is workable even with limited target labels. The proposed bound relates the target risk to source model performance, domain and task differences based on Wasserstein distance. We also extend our bound into unsupervised settings and establish the generalization bound from finite empirical samples. Our experiments in image classification and remaining useful life regression prediction illustrate the effectiveness of the WDJE in determining whether to transfer or not, and the proposed bound in approximating the target transfer risk.