Semi-supervised Vector-valued Learning: Improved Bounds and Algorithms
This work addresses semi-supervised learning for vector-valued outputs, which is incremental as it builds on existing bounds and methods to improve performance in domains like multi-task learning.
The paper tackles the problem of vector-valued learning, such as in multi-task and transfer learning, by deriving improved semi-supervised excess risk bounds using local Rademacher complexity and unlabeled data, achieving convergence rates from square root of labeled sample size to square root of total sample size, and proposes an algorithm that significantly outperforms compared methods in experiments.
Vector-valued learning, where the output space admits a vector-valued structure, is an important problem that covers a broad family of important domains, e.g. multi-task learning and transfer learning. Using local Rademacher complexity and unlabeled data, we derive novel semi-supervised excess risk bounds for general vector-valued learning from both kernel perspective and linear perspective. The derived bounds are much sharper than existing ones and the convergence rates are improved from the square root of labeled sample size to the square root of total sample size or directly dependent on labeled sample size. Motivated by our theoretical analysis, we propose a general semi-supervised algorithm for efficiently learning vector-valued functions, incorporating both local Rademacher complexity and Laplacian regularization. Extensive experimental results illustrate the proposed algorithm significantly outperforms the compared methods, which coincides with our theoretical findings.