Stability of Multi-Task Kernel Regression Algorithms
This work addresses theoretical guarantees for multi-task learning methods, which is incremental as it extends stability analysis to nonlinear and infinite-dimensional settings.
The paper tackles the problem of ensuring stability and generalization for multi-task kernel regression algorithms in reproducing Hilbert spaces with operator-valued kernels, showing they are uniformly stable and deriving generalization bounds and consistency even with non Hilbert-Schmidt kernels.
We study the stability properties of nonlinear multi-task regression in reproducing Hilbert spaces with operator-valued kernels. Such kernels, a.k.a. multi-task kernels, are appropriate for learning prob- lems with nonscalar outputs like multi-task learning and structured out- put prediction. We show that multi-task kernel regression algorithms are uniformly stable in the general case of infinite-dimensional output spaces. We then derive under mild assumption on the kernel generaliza- tion bounds of such algorithms, and we show their consistency even with non Hilbert-Schmidt operator-valued kernels . We demonstrate how to apply the results to various multi-task kernel regression methods such as vector-valued SVR and functional ridge regression.