Improving Topic Relevance Model by Mix-structured Summarization and LLM-based Data Augmentation
This work addresses the problem of enhancing search relevance for social platforms like Dianping, though it is incremental in nature.
The paper tackled the challenges of modeling topic relevance in social search by using query-based and document summaries as model inputs and employing LLMs for data augmentation, resulting in improved performance in offline experiments and online A/B tests.
Topic relevance between query and document is a very important part of social search, which can evaluate the degree of matching between document and user's requirement. In most social search scenarios such as Dianping, modeling search relevance always faces two challenges. One is that many documents in social search are very long and have much redundant information. The other is that the training data for search relevance model is difficult to get, especially for multi-classification relevance model. To tackle above two problems, we first take query concatenated with the query-based summary and the document summary without query as the input of topic relevance model, which can help model learn the relevance degree between query and the core topic of document. Then, we utilize the language understanding and generation abilities of large language model (LLM) to rewrite and generate query from queries and documents in existing training data, which can construct new query-document pairs as training data. Extensive offline experiments and online A/B tests show that the proposed approaches effectively improve the performance of relevance modeling.