Efficient AUC Optimization for Information Ranking Applications
This work addresses the need for better AUC optimization in information retrieval systems, particularly for multi-relevance ranking, but it is incremental as it builds on existing methods with a focus on non-linear techniques.
The paper tackles the problem of optimizing Area under the ROC curve (AUC) for information ranking applications, where traditional methods often optimize error rate instead. It introduces an efficient non-linear approach using additive regression trees, achieving comparable performance on binary-relevance datasets and better results on multi-relevance benchmark datasets.
Adequate evaluation of an information retrieval system to estimate future performance is a crucial task. Area under the ROC curve (AUC) is widely used to evaluate the generalization of a retrieval system. However, the objective function optimized in many retrieval systems is the error rate and not the AUC value. This paper provides an efficient and effective non-linear approach to optimize AUC using additive regression trees, with a special emphasis on the use of multi-class AUC (MAUC) because multiple relevance levels are widely used in many ranking applications. Compared to a conventional linear approach, the performance of the non-linear approach is comparable on binary-relevance benchmark datasets and is better on multi-relevance benchmark datasets.