Limits of Transfer Learning
This work addresses the lack of theoretical development in transfer learning, providing foundational insights that could impact a wide range of machine learning problems.
The paper tackles the theoretical underpinnings of transfer learning by proving novel results on the necessity of selecting appropriate information to transfer and the dependence between transferred information and target problems, establishing an upper bound on improvement based on probabilistic changes in the algorithm.
Transfer learning involves taking information and insight from one problem domain and applying it to a new problem domain. Although widely used in practice, theory for transfer learning remains less well-developed. To address this, we prove several novel results related to transfer learning, showing the need to carefully select which sets of information to transfer and the need for dependence between transferred information and target problems. Furthermore, we prove how the degree of probabilistic change in an algorithm using transfer learning places an upper bound on the amount of improvement possible. These results build on the algorithmic search framework for machine learning, allowing the results to apply to a wide range of learning problems using transfer.