Theoretical Guarantees of Transfer Learning
This is an incremental contribution that provides theoretical insights for researchers in machine learning, addressing the challenge of analyzing transfer learning when target data is scarce.
The paper surveys theoretical works in transfer learning and summarizes key guarantees proving its effectiveness, deriving bounds using model complexity and algorithm stability, and also proves a new generalization bound for multi-source transfer learning using VC-theory that is more informative than previous work.
Transfer learning has been proven effective when within-target labeled data is scarce. A lot of works have developed successful algorithms and empirically observed positive transfer effect that improves target generalization error using source knowledge. However, theoretical analysis of transfer learning is more challenging due to the nature of the problem and thus is less studied. In this report, we do a survey of theoretical works in transfer learning and summarize key theoretical guarantees that prove the effectiveness of transfer learning. The theoretical bounds are derived using model complexity and learning algorithm stability. As we should see, these works exhibit a trade-off between tight bounds and restrictive assumptions. Moreover, we also prove a new generalization bound for the multi-source transfer learning problem using the VC-theory, which is more informative than the one proved in previous work.