Document Similarity for Texts of Varying Lengths via Hidden Topics
This addresses a specific issue in text similarity for applications dealing with documents of varying lengths, but it appears incremental as it builds on existing topic modeling methods.
The paper tackled the problem of measuring similarity between texts of non-comparable lengths, such as a long document and its summary, by proposing a document matching approach using hidden topics. The result showed that the algorithm consistently and widely outperformed strong baselines on two matching tasks.
Measuring similarity between texts is an important task for several applications. Available approaches to measure document similarity are inadequate for document pairs that have non-comparable lengths, such as a long document and its summary. This is because of the lexical, contextual and the abstraction gaps between a long document of rich details and its concise summary of abstract information. In this paper, we present a document matching approach to bridge this gap, by comparing the texts in a common space of hidden topics. We evaluate the matching algorithm on two matching tasks and find that it consistently and widely outperforms strong baselines. We also highlight the benefits of incorporating domain knowledge to text matching.