Momentum-based Gradient Methods in Multi-Objective Recommendation
This work addresses the problem of unstable optimization in multi-objective recommender systems for developers and researchers, representing an incremental improvement by adapting an existing single-objective method.
The paper tackled the brittle behavior of classic multi-gradient descent in multi-objective recommender systems by developing a multi-objective Adamize method that stabilizes gradients using Adam optimizer principles, resulting in significant improvements in Pareto front metrics such as hypervolume, coverage, and spacing, with the Adamized Pareto front strictly dominating previous ones.
Multi-objective gradient methods are becoming the standard for solving multi-objective problems. Among others, they show promising results in developing multi-objective recommender systems with both correlated and conflicting objectives. Classic multi-gradient~descent usually relies on the combination of the gradients, not including the computation of first and second moments of the gradients. This leads to a brittle behavior and misses important areas in the solution space. In this work, we create a multi-objective model-agnostic Adamize method that leverages the benefits of the Adam optimizer in single-objective problems. This corrects and stabilizes~the~gradients of every objective before calculating a common gradient descent vector that optimizes all the objectives simultaneously. We evaluate the benefits of Multi-objective Adamize on two multi-objective recommender systems and for three different objective combinations, both correlated or conflicting. We report significant improvements, measured with three different Pareto front metrics: hypervolume, coverage, and spacing. Finally, we show that the \textit{Adamized} Pareto front strictly dominates the previous one on multiple objective pairs.