Unprejudiced Training Auxiliary Tasks Makes Primary Better: A Multi-Task Learning Perspective
This addresses a bottleneck in multi-task learning for AI practitioners, though it appears incremental as it builds on existing methods.
The paper tackles the problem of insufficient training on auxiliary tasks in multi-task learning, which limits their ability to enhance the primary task, by proposing an uncertainty-based impartial learning method that ensures balanced training and uses gradients and uncertainty information; experiments show it achieves performance comparable to or better than state-of-the-art approaches.
Human beings can leverage knowledge from relative tasks to improve learning on a primary task. Similarly, multi-task learning methods suggest using auxiliary tasks to enhance a neural network's performance on a specific primary task. However, previous methods often select auxiliary tasks carefully but treat them as secondary during training. The weights assigned to auxiliary losses are typically smaller than the primary loss weight, leading to insufficient training on auxiliary tasks and ultimately failing to support the main task effectively. To address this issue, we propose an uncertainty-based impartial learning method that ensures balanced training across all tasks. Additionally, we consider both gradients and uncertainty information during backpropagation to further improve performance on the primary task. Extensive experiments show that our method achieves performance comparable to or better than state-of-the-art approaches. Moreover, our weighting strategy is effective and robust in enhancing the performance of the primary task regardless the noise auxiliary tasks' pseudo labels.