Theoretical Models of Learning to Learn
This work addresses the foundational issue of bias in machine learning for researchers, but it is incremental as it builds on existing theoretical frameworks.
The paper tackles the problem of enabling machines to learn their own bias by introducing two theoretical models of learning to learn, based on empirical process theory and hierarchical Bayes, and presents main theoretical results without specifying concrete numerical outcomes.
A Machine can only learn if it is biased in some way. Typically the bias is supplied by hand, for example through the choice of an appropriate set of features. However, if the learning machine is embedded within an {\em environment} of related tasks, then it can {\em learn} its own bias by learning sufficiently many tasks from the environment. In this paper two models of bias learning (or equivalently, learning to learn) are introduced and the main theoretical results presented. The first model is a PAC-type model based on empirical process theory, while the second is a hierarchical Bayes model.