Efficient and Effective Query Auto-Completion
This work addresses the need for more effective QAC systems in large-scale search applications like eBay, though it appears incremental as it builds on existing data structures.
The paper tackled the problem of Query Auto-Completion (QAC) systems being limited to prefix-based completions, which reduces effectiveness and can cause monetary losses in applications like web search and e-commerce, by implementing a new system at eBay that combines an inverted index with succinct data structures, achieving improved efficiency and effectiveness compared to prior approaches.
Query Auto-Completion (QAC) is an ubiquitous feature of modern textual search systems, suggesting possible ways of completing the query being typed by the user. Efficiency is crucial to make the system have a real-time responsiveness when operating in the million-scale search space. Prior work has extensively advocated the use of a trie data structure for fast prefix-search operations in compact space. However, searching by prefix has little discovery power in that only completions that are prefixed by the query are returned. This may impact negatively the effectiveness of the QAC system, with a consequent monetary loss for real applications like Web Search Engines and eCommerce. In this work we describe the implementation that empowers a new QAC system at eBay, and discuss its efficiency/effectiveness in relation to other approaches at the state-of-the-art. The solution is based on the combination of an inverted index with succinct data structures, a much less explored direction in the literature. This system is replacing the previous implementation based on Apache SOLR that was not always able to meet the required service-level-agreement.