ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search
This addresses the problem of limited access to click data for researchers in information retrieval, enabling studies on query mining and ranking, though it is incremental as it builds on existing TREC corpora.
The paper tackles the lack of publicly available click logs for academic research by releasing ORCAS, a dataset of 18 million clicked query-document pairs derived from aggregated and filtered search engine logs, which includes 10 million distinct queries and 1.4 million URLs, and preliminary experiments show it provides 28x more queries and 49x more connections compared to existing TREC DL training data.
Users of Web search engines reveal their information needs through queries and clicks, making click logs a useful asset for information retrieval. However, click logs have not been publicly released for academic use, because they can be too revealing of personally or commercially sensitive information. This paper describes a click data release related to the TREC Deep Learning Track document corpus. After aggregation and filtering, including a k-anonymity requirement, we find 1.4 million of the TREC DL URLs have 18 million connections to 10 million distinct queries. Our dataset of these queries and connections to TREC documents is of similar size to proprietary datasets used in previous papers on query mining and ranking. We perform some preliminary experiments using the click data to augment the TREC DL training data, offering by comparison: 28x more queries, with 49x more connections to 4.4x more URLs in the corpus. We present a description of the dataset's generation process, characteristics, use in ranking and suggest other potential uses.