Shopping Queries Dataset: A Large-Scale ESCI Benchmark for Improving Product Search
This dataset addresses the problem of enhancing search engine user experience for e-commerce, but it is incremental as it provides a new benchmark rather than a novel method.
The paper tackles the challenge of improving product search quality by introducing the Shopping Queries Dataset, a large-scale multilingual benchmark with 130,000 unique queries and 2.6 million manually labeled relevance judgments, and presents baseline results for three evaluation tasks.
Improving the quality of search results can significantly enhance users experience and engagement with search engines. In spite of several recent advancements in the fields of machine learning and data mining, correctly classifying items for a particular user search query has been a long-standing challenge, which still has a large room for improvement. This paper introduces the "Shopping Queries Dataset", a large dataset of difficult Amazon search queries and results, publicly released with the aim of fostering research in improving the quality of search results. The dataset contains around 130 thousand unique queries and 2.6 million manually labeled (query,product) relevance judgements. The dataset is multilingual with queries in English, Japanese, and Spanish. The Shopping Queries Dataset is being used in one of the KDDCup'22 challenges. In this paper, we describe the dataset and present three evaluation tasks along with baseline results: (i) ranking the results list, (ii) classifying product results into relevance categories, and (iii) identifying substitute products for a given query. We anticipate that this data will become the gold standard for future research in the topic of product search.