Matúš Medo

Yuhao Zhou, Ruijie Wang, Yi-Cheng Zhang et al.

While PageRank has been extensively used to rank sport tournament participants (teams or individuals), its superiority over simpler ranking methods has been never clearly demonstrated. We use sports results from 18 major leagues to calibrate a state-of-art model for synthetic sports results. Model data are then used to assess the ranking performance of PageRank in a controlled setting. We find that PageRank outperforms the benchmark ranking by the number of wins only when a small fraction of all games have been played. Increased randomness in the data, such as intrinsic randomness of outcomes or advantage of home teams, further reduces the range of PageRank's superiority. We propose a new PageRank variant which outperforms PageRank in all evaluated settings, yet shares its sensitivity to increased randomness in the data. Our main findings are confirmed by evaluating the ranking algorithms on real data. Our work demonstrates the danger of using novel metrics and algorithms without considering their limits of applicability.

Shuqi Xu, Manuel Sebastian Mariani, Linyuan Lü et al.

Despite the increasing use of citation-based metrics for research evaluation purposes, we do not know yet which metrics best deliver on their promise to gauge the significance of a scientific paper or a patent. We assess 17 network-based metrics by their ability to identify milestone papers and patents in three large citation datasets. We find that traditional information-retrieval evaluation metrics are strongly affected by the interplay between the age distribution of the milestone items and age biases of the evaluated metrics. Outcomes of these metrics are therefore not representative of the metrics' ranking ability. We argue in favor of a modified evaluation procedure that explicitly penalizes biased metrics and allows us to reveal metrics' performance patterns that are consistent across the datasets. PageRank and LeaderRank turn out to be the best-performing ranking metrics when their age bias is suppressed by a simple transformation of the scores that they produce, whereas other popular metrics, including citation count, HITS and Collective Influence, produce significantly worse ranking results.

Shilun Zhang, Matúš Medo, Linyuan Lü et al.

When we search online for content, we are constantly exposed to rankings. For example, web search results are presented as a ranking, and online bookstores often show us lists of best-selling books. While popularity-based ranking algorithms (like Google's PageRank) have been extensively studied in previous works, we still lack a clear understanding of their potential systemic consequences. In this work, we fill this gap by introducing a new model of network growth that allows us to compare the properties of the networks generated under the influence of different ranking algorithms. We show that by correcting for the omnipresent age bias of popularity-based ranking algorithms, the resulting networks exhibit a significantly larger agreement between the nodes' inherent quality and their long-term popularity, and a less concentrated popularity distribution. To further promote popularity diversity, we introduce and validate a perturbation of the original rankings where a small number of randomly-selected nodes are promoted to the top of the ranking. Our findings move the first steps toward a model-based understanding of the long-term impact of popularity-based ranking algorithms, and could be used as an informative tool for the design of improved information filtering tools.