Multi-task Bias-Variance Trade-off Through Functional Constraints
This addresses the challenge of balancing task-specific and shared learning in multi-task settings, though it appears incremental as it builds on existing bias-variance trade-off concepts.
The paper tackles the bias-variance trade-off in multi-task learning by proposing a constrained formulation that enforces domain-specific solutions to be close to a central function, with experimental results showing it outperforms both task-specific and single classifiers on a real multi-domain classification problem.
Multi-task learning aims to acquire a set of functions, either regressors or classifiers, that perform well for diverse tasks. At its core, the idea behind multi-task learning is to exploit the intrinsic similarity across data sources to aid in the learning process for each individual domain. In this paper we draw intuition from the two extreme learning scenarios -- a single function for all tasks, and a task-specific function that ignores the other tasks dependencies -- to propose a bias-variance trade-off. To control the relationship between the variance (given by the number of i.i.d. samples), and the bias (coming from data from other task), we introduce a constrained learning formulation that enforces domain specific solutions to be close to a central function. This problem is solved in the dual domain, for which we propose a stochastic primal-dual algorithm. Experimental results for a multi-domain classification problem with real data show that the proposed procedure outperforms both the task specific, as well as the single classifiers.