Standardization of Weighted Ranking Correlation Coefficients
This addresses a statistical issue for researchers using weighted ranking correlations in contexts like information retrieval or recommendation systems, but it is incremental as it builds on existing weighted coefficients.
The paper tackles the problem of weighted ranking correlation coefficients having non-zero expected values for random rankings, which undermines the concept of uncorrelation. It proposes a standardization function to map these coefficients to a form with zero expected value while preserving their statistical properties.
A relevant problem in statistics is defining the correlation of two rankings of a list of items. Kendall's tau and Spearman's rho are two well established correlation coefficients, characterized by a symmetric form that ensures zero expected value between two pairs of rankings randomly chosen with uniform probability. However, in recent years, several weighted versions of the original Spearman and Kendall coefficients have emerged that take into account the greater importance of top ranks compared to low ranks, which is common in many contexts. The weighting schemes break the symmetry, causing a non-zero expected value between two random rankings. This issue is very relevant, as it undermines the concept of uncorrelation between rankings. In this paper, we address this problem by proposing a standardization function $g(x)$ that maps a correlation ranking coefficient $Γ$ in a standard form $g(Γ)$ that has zero expected value, while maintaining the relevant statistical properties of $Γ$.