CORG: Generating Answers from Complex, Interrelated Contexts
This addresses the challenge of handling inconsistent and interrelated contexts in knowledge-intensive tasks for natural language processing applications, representing an incremental improvement over prior methods.
The paper tackles the problem of language models struggling with complex, interrelated contexts in real-world corpora, where knowledge recurs with inconsistencies like ambiguous naming or errors. It introduces the Context Organizer (CORG) framework, which organizes contexts into groups for independent processing, and results show it outperforms existing grouping methods and achieves comparable performance to more intensive single-context approaches.
In a real-world corpus, knowledge frequently recurs across documents but often contains inconsistencies due to ambiguous naming, outdated information, or errors, leading to complex interrelationships between contexts. Previous research has shown that language models struggle with these complexities, typically focusing on single factors in isolation. We classify these relationships into four types: distracting, ambiguous, counterfactual, and duplicated. Our analysis reveals that no single approach effectively addresses all these interrelationships simultaneously. Therefore, we introduce Context Organizer (CORG), a framework that organizes multiple contexts into independently processed groups. This design allows the model to efficiently find all relevant answers while ensuring disambiguation. CORG consists of three key components: a graph constructor, a reranker, and an aggregator. Our results demonstrate that CORG balances performance and efficiency effectively, outperforming existing grouping methods and achieving comparable results to more computationally intensive, single-context approaches.