Patterns for Learning with Side Information
It offers a unifying perspective for researchers working on related learning paradigms, but is incremental as it connects existing approaches rather than introducing new methods.
The paper tackles the problem of improving generalization in machine learning by incorporating side information, showing that this approach unifies methods like multi-task and multi-view learning, and provides experimental evaluation in supervised tasks.
Supervised, semi-supervised, and unsupervised learning estimate a function given input/output samples. Generalization of the learned function to unseen data can be improved by incorporating side information into learning. Side information are data that are neither from the input space nor from the output space of the function, but include useful information for learning it. In this paper we show that learning with side information subsumes a variety of related approaches, e.g. multi-task learning, multi-view learning and learning using privileged information. Our main contributions are (i) a new perspective that connects these previously isolated approaches, (ii) insights about how these methods incorporate different types of prior knowledge, and hence implement different patterns, (iii) facilitating the application of these methods in novel tasks, as well as (iv) a systematic experimental evaluation of these patterns in two supervised learning tasks.