Overview of the TREC 2023 Product Product Search Track
This work addresses product search retrieval for researchers and practitioners, but it is incremental as it builds on existing TREC tracks with a new domain-specific focus.
The TREC 2023 Product Search Track introduced a new reusable collection to evaluate how metadata and multi-modal data affect retrieval accuracy, finding that traditional retrieval systems often outperform pretrained embedding models in product search, with dense retrieval methods frequently underperforming.
This is the first year of the TREC Product search track. The focus this year was the creation of a reusable collection and evaluation of the impact of the use of metadata and multi-modal data on retrieval accuracy. This year we leverage the new product search corpus, which includes contextual metadata. Our analysis shows that in the product search domain, traditional retrieval systems are highly effective and commonly outperform general-purpose pretrained embedding models. Our analysis also evaluates the impact of using simplified and metadata-enhanced collections, finding no clear trend in the impact of the expanded collection. We also see some surprising outcomes; despite their widespread adoption and competitive performance on other tasks, we find single-stage dense retrieval runs can commonly be noncompetitive or generate low-quality results both in the zero-shot and fine-tuned domain.