Declarative Techniques for NL Queries over Heterogeneous Data
This addresses the challenge of practical natural language querying for users in industrial environments where data is diverse and fragmented, representing an incremental improvement over existing methods.
The paper tackles the problem of answering natural language queries across heterogeneous data sources (databases and APIs) in industrial settings, and demonstrates that their declarative approach significantly outperforms state-of-the-art LLM-based systems on new benchmarks simulating real-world heterogeneity.
In many industrial settings, users wish to ask questions in natural language, the answers to which require assembling information from diverse structured data sources. With the advent of Large Language Models (LLMs), applications can now translate natural language questions into a set of API calls or database calls, execute them, and combine the results into an appropriate natural language response. However, these applications remain impractical in realistic industrial settings because they do not cope with the data source heterogeneity that typifies such environments. In this work, we simulate the heterogeneity of real industry settings by introducing two extensions of the popular Spider benchmark dataset that require a combination of database and API calls. Then, we introduce a declarative approach to handling such data heterogeneity and demonstrate that it copes with data source heterogeneity significantly better than state-of-the-art LLM-based agentic or imperative code generation systems. Our augmented benchmarks are available to the research community.