Studying Ranking-Incentivized Web Dynamics
This provides initial empirical evidence for ranking-incentivized web dynamics, addressing a lack of public datasets, but it is incremental as it extends known theoretical findings to web data.
The authors tackled the problem of empirically analyzing how web page authors respond to search rankings by creating a dataset from TREC's ClueWeb09 and the Internet Archive, finding that authors mimic highly ranked content, which can improve future rankings.
The ranking incentives of many authors of Web pages play an important role in the Web dynamics. That is, authors who opt to have their pages highly ranked for queries of interest, often respond to rankings for these queries by manipulating their pages; the goal is to improve the pages' future rankings. Various theoretical aspects of this dynamics have recently been studied using game theory. However, empirical analysis of the dynamics is highly constrained due to lack of publicly available datasets.We present an initial such dataset that is based on TREC's ClueWeb09 dataset. Specifically, we used the WayBack Machine of the Internet Archive to build a document collection that contains past snapshots of ClueWeb documents which are highly ranked by some initial search performed for ClueWeb queries. Temporal analysis of document changes in this dataset reveals that findings recently presented for small-scale controlled ranking competitions between documents' authors also hold for Web data. Specifically, documents' authors tend to mimic the content of documents that were highly ranked in the past, and this practice can result in improved ranking.