Learning Bound for Parameter Transfer Learning
This addresses the need for theoretical foundations in transfer learning, particularly for self-taught learning, which has shown empirical success but lacked analysis, making it incremental in providing formal guarantees.
The paper tackles the problem of deriving theoretical learning bounds for parameter transfer learning, introducing concepts like local stability and parameter transfer learnability, and applies this to provide the first theoretical bound for self-taught learning.
We consider a transfer-learning problem by using the parameter transfer approach, where a suitable parameter of feature mapping is learned through one task and applied to another objective task. Then, we introduce the notion of the local stability and parameter transfer learnability of parametric feature mapping,and thereby derive a learning bound for parameter transfer algorithms. As an application of parameter transfer learning, we discuss the performance of sparse coding in self-taught learning. Although self-taught learning algorithms with plentiful unlabeled data often show excellent empirical performance, their theoretical analysis has not been studied. In this paper, we also provide the first theoretical learning bound for self-taught learning.