Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approach
This addresses a specific bottleneck in multi-objective learning for researchers and practitioners, offering a provably convergent solution, though it appears incremental relative to existing methods.
The paper tackles the problem of gradient bias in stochastic multi-objective optimization methods, which degrades performance in tasks like multi-task learning. It proposes a new method called MoCo that provably converges without increasing batch size, demonstrating effectiveness in simulations on supervised and reinforcement learning.
Machine learning problems with multiple objective functions appear either in learning with multiple criteria where learning has to make a trade-off between multiple performance metrics such as fairness, safety and accuracy; or, in multi-task learning where multiple tasks are optimized jointly, sharing inductive bias between them. This problems are often tackled by the multi-objective optimization framework. However, existing stochastic multi-objective gradient methods and its variants (e.g., MGDA, PCGrad, CAGrad, etc.) all adopt a biased noisy gradient direction, which leads to degraded empirical performance. To this end, we develop a stochastic Multi-objective gradient Correction (MoCo) method for multi-objective optimization. The unique feature of our method is that it can guarantee convergence without increasing the batch size even in the non-convex setting. Simulations on multi-task supervised and reinforcement learning demonstrate the effectiveness of our method relative to state-of-the-art methods.