Multi-Task Cooperative Learning via Searching for Flat Minima
This addresses the negative transfer problem in MTL for medical image analysis, though it appears incremental as it builds on existing MTL optimization methods.
The paper tackles the problem of competitive feature learning in multi-task learning (MTL) for medical image analysis by formulating MTL as a multi/bi-level optimization problem to enable cooperative learning, and it shows promising results compared to state-of-the-art MTL approaches on three public datasets.
Multi-task learning (MTL) has shown great potential in medical image analysis, improving the generalizability of the learned features and the performance in individual tasks. However, most of the work on MTL focuses on either architecture design or gradient manipulation, while in both scenarios, features are learned in a competitive manner. In this work, we propose to formulate MTL as a multi/bi-level optimization problem, and therefore force features to learn from each task in a cooperative approach. Specifically, we update the sub-model for each task alternatively taking advantage of the learned sub-models of the other tasks. To alleviate the negative transfer problem during the optimization, we search for flat minima for the current objective function with regard to features from other tasks. To demonstrate the effectiveness of the proposed approach, we validate our method on three publicly available datasets. The proposed method shows the advantage of cooperative learning, and yields promising results when compared with the state-of-the-art MTL approaches. The code will be available online.