Convex Multitask Learning with Flexible Task Clusters
This addresses the issue of negative transfer in multitask learning for applications where task relationships vary across features, representing an incremental improvement over existing cluster-based methods.
The paper tackles the problem of negative transfer in multitask learning by proposing a novel formulation that captures task relationships at the feature-level, allowing different task clusters for different features without pre-specifying cluster numbers, and experiments show it consistently achieves high accuracy and aligns with known task structures.
Traditionally, multitask learning (MTL) assumes that all the tasks are related. This can lead to negative transfer when tasks are indeed incoherent. Recently, a number of approaches have been proposed that alleviate this problem by discovering the underlying task clusters or relationships. However, they are limited to modeling these relationships at the task level, which may be restrictive in some applications. In this paper, we propose a novel MTL formulation that captures task relationships at the feature-level. Depending on the interactions among tasks and features, the proposed method construct different task clusters for different features, without even the need of pre-specifying the number of clusters. Computationally, the proposed formulation is strongly convex, and can be efficiently solved by accelerated proximal methods. Experiments are performed on a number of synthetic and real-world data sets. Under various degrees of task relationships, the accuracy of the proposed method is consistently among the best. Moreover, the feature-specific task clusters obtained agree with the known/plausible task structures of the data.