"Piaf" vs "Adele": classifying encyclopedic queries using automatically labeled training data
This addresses the need for efficient query intent classification in search engines, though it is incremental as it builds on existing methods for feature extraction and labeling.
The paper tackled the problem of classifying web queries as encyclopedic by training a classifier using automatically labeled data from query logs, achieving an F1 score above 87% that competes with a Google-based baseline.
Encyclopedic queries express the intent of obtaining information typically available in encyclopedias, such as biographical, geographical or historical facts. In this paper, we train a classifier for detecting the encyclopedic intent of web queries. For training such a classifier, we automatically label training data from raw query logs. We use click-through data to select positive examples of encyclopedic queries as those queries that mostly lead to Wikipedia articles. We investigated a large set of features that can be generated to describe the input query. These features include both term-specific patterns as well as query projections on knowledge bases items (e.g. Freebase). Results show that using these feature sets it is possible to achieve an F1 score above 87%, competing with a Google-based baseline, which uses a much wider set of signals to boost the ranking of Wikipedia for potential encyclopedic queries. The results also show that both query projections on Wikipedia article titles and Freebase entity match represent the most relevant groups of features. When the training set contains frequent positive examples (i.e rare queries are excluded) results tend to improve.