New Tight Relaxations of Rank Minimization for Multi-Task Learning
This work addresses the challenge of learning shared low-rank subspaces in multi-task learning, offering incremental improvements in optimization techniques for this domain-specific problem.
The paper tackles the NP-hard problem of exact rank minimization in multi-task learning by proposing two novel regularization terms that approximate rank more tightly than trace norm, and introduces a re-weighted iterative strategy to solve them, achieving superior performance over existing methods on benchmark datasets.
Multi-task learning has been observed by many researchers, which supposes that different tasks can share a low-rank common yet latent subspace. It means learning multiple tasks jointly is better than learning them independently. In this paper, we propose two novel multi-task learning formulations based on two regularization terms, which can learn the optimal shared latent subspace by minimizing the exactly $k$ minimal singular values. The proposed regularization terms are the more tight approximations of rank minimization than trace norm. But it's an NP-hard problem to solve the exact rank minimization problem. Therefore, we design a novel re-weighted based iterative strategy to solve our models, which can tactically handle the exact rank minimization problem by setting a large penalizing parameter. Experimental results on benchmark datasets demonstrate that our methods can correctly recover the low-rank structure shared across tasks, and outperform related multi-task learning methods.