Auxiliary Learning as an Asymmetric Bargaining Game
This work addresses the challenge of optimizing multiple objectives in auxiliary learning for machine learning practitioners, offering a novel method to enhance model performance, though it is incremental as it builds on existing auxiliary learning frameworks.
The paper tackled the problem of balancing tasks in auxiliary learning to improve generalization, particularly with small datasets, by proposing AuxiNash, which formalizes it as an asymmetric bargaining game and learns task bargaining power, resulting in consistent outperformance over competing methods on multiple benchmarks.
Auxiliary learning is an effective method for enhancing the generalization capabilities of trained models, particularly when dealing with small datasets. However, this approach may present several difficulties: (i) optimizing multiple objectives can be more challenging, and (ii) how to balance the auxiliary tasks to best assist the main task is unclear. In this work, we propose a novel approach, named AuxiNash, for balancing tasks in auxiliary learning by formalizing the problem as generalized bargaining game with asymmetric task bargaining power. Furthermore, we describe an efficient procedure for learning the bargaining power of tasks based on their contribution to the performance of the main task and derive theoretical guarantees for its convergence. Finally, we evaluate AuxiNash on multiple multi-task benchmarks and find that it consistently outperforms competing methods.