Lower-Left Partial AUC: An Effective and Efficient Optimization Metric for Recommendation
This addresses the need for scalable optimization metrics in recommendation systems, offering a practical solution that balances accuracy and efficiency, though it is incremental in improving upon existing metrics.
The paper tackles the problem of finding an effective and efficient optimization metric for recommendation systems by proposing Lower-Left Partial AUC (LLPAUC), which achieves strong correlation with Top-K ranking metrics while being computationally efficient like AUC, as validated on three datasets.
Optimization metrics are crucial for building recommendation systems at scale. However, an effective and efficient metric for practical use remains elusive. While Top-K ranking metrics are the gold standard for optimization, they suffer from significant computational overhead. Alternatively, the more efficient accuracy and AUC metrics often fall short of capturing the true targets of recommendation tasks, leading to suboptimal performance. To overcome this dilemma, we propose a new optimization metric, Lower-Left Partial AUC (LLPAUC), which is computationally efficient like AUC but strongly correlates with Top-K ranking metrics. Compared to AUC, LLPAUC considers only the partial area under the ROC curve in the Lower-Left corner to push the optimization focus on Top-K. We provide theoretical validation of the correlation between LLPAUC and Top-K ranking metrics and demonstrate its robustness to noisy user feedback. We further design an efficient point-wise recommendation loss to maximize LLPAUC and evaluate it on three datasets, validating its effectiveness and robustness.