PSL: Rethinking and Improving Softmax Loss from Pairwise Perspective for Recommendation
It addresses incremental improvements in loss functions for recommender systems, potentially enhancing ranking accuracy and handling false negatives better.
This work tackles limitations in Softmax Loss for recommender systems by analyzing it from a pairwise perspective and proposing Pairwise Softmax Loss (PSL), which replaces the exponential function with other activation functions to improve ranking metrics and robustness, with empirical validation showing effectiveness.
Softmax Loss (SL) is widely applied in recommender systems (RS) and has demonstrated effectiveness. This work analyzes SL from a pairwise perspective, revealing two significant limitations: 1) the relationship between SL and conventional ranking metrics like DCG is not sufficiently tight; 2) SL is highly sensitive to false negative instances. Our analysis indicates that these limitations are primarily due to the use of the exponential function. To address these issues, this work extends SL to a new family of loss functions, termed Pairwise Softmax Loss (PSL), which replaces the exponential function in SL with other appropriate activation functions. While the revision is minimal, we highlight three merits of PSL: 1) it serves as a tighter surrogate for DCG with suitable activation functions; 2) it better balances data contributions; and 3) it acts as a specific BPR loss enhanced by Distributionally Robust Optimization (DRO). We further validate the effectiveness and robustness of PSL through empirical experiments. The code is available at https://github.com/Tiny-Snow/IR-Benchmark.