More Flexible PAC-Bayesian Meta-Learning by Learning Learning Algorithms
This provides a more flexible theoretical foundation for meta-learning, potentially benefiting researchers and practitioners in AI by enabling better analysis and design of meta-learning methods.
The paper tackles the problem of meta-learning by introducing a PAC-Bayesian framework that allows direct learning of algorithms for knowledge transfer between tasks, and it empirically shows improved prediction quality in practical mechanisms.
We introduce a new framework for studying meta-learning methods using PAC-Bayesian theory. Its main advantage over previous work is that it allows for more flexibility in how the transfer of knowledge between tasks is realized. For previous approaches, this could only happen indirectly, by means of learning prior distributions over models. In contrast, the new generalization bounds that we prove express the process of meta-learning much more directly as learning the learning algorithm that should be used for future tasks. The flexibility of our framework makes it suitable to analyze a wide range of meta-learning mechanisms and even design new mechanisms. Other than our theoretical contributions we also show empirically that our framework improves the prediction quality in practical meta-learning mechanisms.