Sum of rank ratios: an alternative to percentiles for research assessment, from groundbreaking to mainstream research
For bibliometricians and research evaluators, the Rn index offers a simple, robust alternative to percentile-based metrics that better captures the quality of science produced by countries with varying levels of groundbreaking research.
The paper proposes the Rn index, a rank-based metric using the sum of rank ratios, to assess research output across all impact levels, from groundbreaking to mainstream. It demonstrates that the Rn index outperforms traditional indicators like top 10% or top 1% highly cited papers for evaluating countries that produce few high-impact papers.
Assessing research that pushes the boundaries of knowledge is challenging because such work is extremely infrequent, accounting for only about 0.01 per cent of all research outputs. Consequently, knowledge about how to evaluate this type of research is far more limited than the well established methods used to assess more common research outcomes. This study addresses this gap by using a rank based approach in which each paper is assigned a unique value equal to the ratio between its local and global ranks. The cumulative value of these ratios, starting from the most cited paper, provides the evaluative basis, and the Rn index described here, using 10 rank ratios, appears to be the best option. Although research assessment based on global ranks was originally developed to evaluate the largest contributors to groundbreaking knowledge, namely, the USA and China, which account for most of the most cited papers, the Rn index has broader applications. This study demonstrates that it is also a better option than the number of top 10 per cent or top 1per cent highly cited papers, which are the most common indicators used to evaluate countries that seldom or never produce cutting-edge research that pushes the boundaries of knowledge. In all cases, the Rn index reflects the highest quality science produced by each country. Furthermore, the Rn index can be easily calculated without specialized training in bibliometrics and is insignificantly affected by ties in citation counts.