Follow the bisector: a simple method for multi-objective optimization
This method addresses multi-objective optimization problems, such as those with multi-scale losses, but appears incremental as it builds on existing gradient-based approaches without claiming broad breakthroughs.
The study tackled multi-objective optimization by proposing the Equiangular Direction Method (EDM), which computes descent directions to ensure equal relative decreases in differentiable losses, and tested it on imbalanced classification and multi-task learning using standard datasets.
This study presents a novel Equiangular Direction Method (EDM) to solve a multi-objective optimization problem. We consider optimization problems, where multiple differentiable losses have to be minimized. The presented method computes descent direction in every iteration to guarantee equal relative decrease of objective functions. This descent direction is based on the normalized gradients of the individual losses. Therefore, it is appropriate to solve multi-objective optimization problems with multi-scale losses. We test the proposed method on the imbalanced classification problem and multi-task learning problem, where standard datasets are used. EDM is compared with other methods to solve these problems.