Using Local Knowledge Graph Construction to Scale Seq2Seq Models to Multi-Document Inputs
This addresses the challenge of handling long and diverse web results in open-domain NLP tasks, offering a domain-specific improvement for generative tasks like question answering and summarization.
The paper tackled the problem of scaling sequence-to-sequence models to multi-document inputs for query-based NLP tasks by constructing a local knowledge graph to compress web search information, achieving better performance than using retrieved text portions for long-form question answering and multi-document summarization.
Query-based open-domain NLP tasks require information synthesis from long and diverse web results. Current approaches extractively select portions of web text as input to Sequence-to-Sequence models using methods such as TF-IDF ranking. We propose constructing a local graph structured knowledge base for each query, which compresses the web search information and reduces redundancy. We show that by linearizing the graph into a structured input sequence, models can encode the graph representations within a standard Sequence-to-Sequence setting. For two generative tasks with very long text input, long-form question answering and multi-document summarization, feeding graph representations as input can achieve better performance than using retrieved text portions.