Database Reasoning Over Text
This addresses a limitation in neural models for supporting complex database queries over text, which is incremental by improving scalability and accuracy for tasks like information extraction and question answering.
The paper tackles the problem of answering database-style queries over natural language text, which requires reasoning over sets of facts with operations like join and aggregation. The proposed modular architecture scales to thousands of facts, increasing answer accuracy from 85% to 90% on small databases and maintaining performance on larger ones where baselines fail.
Neural models have shown impressive performance gains in answering queries from natural language text. However, existing works are unable to support database queries, such as "List/Count all female athletes who were born in 20th century", which require reasoning over sets of relevant facts with operations such as join, filtering and aggregation. We show that while state-of-the-art transformer models perform very well for small databases, they exhibit limitations in processing noisy data, numerical operations, and queries that aggregate facts. We propose a modular architecture to answer these database-style queries over multiple spans from text and aggregating these at scale. We evaluate the architecture using WikiNLDB, a novel dataset for exploring such queries. Our architecture scales to databases containing thousands of facts whereas contemporary models are limited by how many facts can be encoded. In direct comparison on small databases, our approach increases overall answer accuracy from 85% to 90%. On larger databases, our approach retains its accuracy whereas transformer baselines could not encode the context.