Curriculum Learning of Multiple Tasks
This work addresses the challenge of improving generalization in multi-task learning for scenarios with varying task relatedness, offering an incremental advance by optimizing task sequences.
The paper tackles the problem of multi-task learning where tasks are not equally related, proposing a sequential approach with sharing between subsequent tasks and a curriculum learning method to find the optimal task order based on a generalization bound criterion. Experimental results show that sequential learning can be more effective than joint learning, task order affects performance, and the model can automatically discover favorable orders.
Sharing information between multiple tasks enables algorithms to achieve good generalization performance even from small amounts of training data. However, in a realistic scenario of multi-task learning not all tasks are equally related to each other, hence it could be advantageous to transfer information only between the most related tasks. In this work we propose an approach that processes multiple tasks in a sequence with sharing between subsequent tasks instead of solving all tasks jointly. Subsequently, we address the question of curriculum learning of tasks, i.e. finding the best order of tasks to be learned. Our approach is based on a generalization bound criterion for choosing the task order that optimizes the average expected classification performance over all tasks. Our experimental results show that learning multiple related tasks sequentially can be more effective than learning them jointly, the order in which tasks are being solved affects the overall performance, and that our model is able to automatically discover the favourable order of tasks.