Multi-Task Learning with Deep Neural Networks: A Survey
It addresses the problem of designing and optimizing shared models for multiple tasks, which is incremental as it compiles existing research without introducing new methods.
This survey provides an overview of multi-task learning methods for deep neural networks, summarizing established and recent directions in architectures, optimization, and task relationship learning, along with common benchmarks.
Multi-task learning (MTL) is a subfield of machine learning in which multiple tasks are simultaneously learned by a shared model. Such approaches offer advantages like improved data efficiency, reduced overfitting through shared representations, and fast learning by leveraging auxiliary information. However, the simultaneous learning of multiple tasks presents new design and optimization challenges, and choosing which tasks should be learned jointly is in itself a non-trivial problem. In this survey, we give an overview of multi-task learning methods for deep neural networks, with the aim of summarizing both the well-established and most recent directions within the field. Our discussion is structured according to a partition of the existing deep MTL techniques into three groups: architectures, optimization methods, and task relationship learning. We also provide a summary of common multi-task benchmarks.