A PAC-Bayesian bound for Lifelong Learning
This work provides a theoretical foundation for lifelong learning, addressing a gap in understanding for researchers and practitioners in machine learning, though it is incremental as it builds on existing paradigms.
The authors tackled the lack of theoretical understanding in lifelong learning by deriving a PAC-Bayesian generalization bound that unifies existing transfer learning paradigms, and they developed two algorithms that achieve results comparable to existing methods.
Transfer learning has received a lot of attention in the machine learning community over the last years, and several effective algorithms have been developed. However, relatively little is known about their theoretical properties, especially in the setting of lifelong learning, where the goal is to transfer information to tasks for which no data have been observed so far. In this work we study lifelong learning from a theoretical perspective. Our main result is a PAC-Bayesian generalization bound that offers a unified view on existing paradigms for transfer learning, such as the transfer of parameters or the transfer of low-dimensional representations. We also use the bound to derive two principled lifelong learning algorithms, and we show that these yield results comparable with existing methods.