StochasticRank: Global Optimization of Scale-Free Discrete Functions
This addresses the challenge of optimizing discrete ranking functions for machine learning applications, though it appears incremental as it builds on existing smoothing and boosting techniques.
The paper tackles the ill-posed problem of directly optimizing ranking metrics by introducing a framework with stochastic smoothing and a novel gradient estimate, which outperforms existing approaches on learning-to-rank datasets and applies to any scale-free discrete loss function.
In this paper, we introduce a powerful and efficient framework for direct optimization of ranking metrics. The problem is ill-posed due to the discrete structure of the loss, and to deal with that, we introduce two important techniques: stochastic smoothing and novel gradient estimate based on partial integration. We show that classic smoothing approaches may introduce bias and present a universal solution for a proper debiasing. Importantly, we can guarantee global convergence of our method by adopting a recently proposed Stochastic Gradient Langevin Boosting algorithm. Our algorithm is implemented as a part of the CatBoost gradient boosting library and outperforms the existing approaches on several learning-to-rank datasets. In addition to ranking metrics, our framework applies to any scale-free discrete loss function.