Learning Model Bias
This addresses the challenge of efficient multi-task learning for AI practitioners, though it is incremental as it builds on existing theoretical frameworks.
The paper tackles the problem of learning domain-specific bias by training on multiple related tasks, showing theoretically that the required examples per task scale as O(a + b/n) when tasks share a representation, and provides experimental support.
In this paper the problem of {\em learning} appropriate domain-specific bias is addressed. It is shown that this can be achieved by learning many related tasks from the same domain, and a theorem is given bounding the number tasks that must be learnt. A corollary of the theorem is that if the tasks are known to possess a common {\em internal representation} or {\em preprocessing} then the number of examples required per task for good generalisation when learning $n$ tasks simultaneously scales like $O(a + \frac{b}{n})$, where $O(a)$ is a bound on the minimum number of examples required to learn a single task, and $O(a + b)$ is a bound on the number of examples required to learn each task independently. An experiment providing strong qualitative support for the theoretical results is reported.