Subtopic-aware View Sampling and Temporal Aggregation for Long-form Document Matching
This work addresses document matching for applications such as news and legal retrieval, but it is incremental as it builds on existing view-based methods with a new aggregation technique.
The paper tackles the problem of long-form document matching by addressing the heterogeneity of matching signals across multiple subtopics, proposing a framework that constructs multiple document views and uses temporal aggregation, achieving effectiveness on tasks like news duplication and legal case retrieval.
Long-form document matching aims to judge the relevance between two documents and has been applied to various scenarios. Most existing works utilize hierarchical or long context models to process documents, which achieve coarse understanding but may ignore details. Some researchers construct a document view with similar sentences about aligned document subtopics to focus on detailed matching signals. However, a long document generally contains multiple subtopics. The matching signals are heterogeneous from multiple topics. Considering only the homologous aligned subtopics may not be representative enough and may cause biased modeling. In this paper, we introduce a new framework to model representative matching signals. First, we propose to capture various matching signals through subtopics of document pairs. Next, We construct multiple document views based on subtopics to cover heterogeneous and valuable details. However, existing spatial aggregation methods like attention, which integrate all these views simultaneously, are hard to integrate heterogeneous information. Instead, we propose temporal aggregation, which effectively integrates different views gradually as the training progresses. Experimental results show that our learning framework is effective on several document-matching tasks, including news duplication and legal case retrieval.