Searching for Better Database Queries in the Outputs of Semantic Parsers
This addresses the challenge of ambiguity and imprecision in semantic parsing for database queries, particularly for unseen databases, though it is incremental as it builds on existing state-of-the-art parsers.
The paper tackles the problem of generating accurate database queries from natural language questions, especially when generalizing to unseen databases, by augmenting neural autoregressive models with a search algorithm that uses an external criterion to evaluate queries. The result shows that this approach finds many queries passing all tests on different datasets.
The task of generating a database query from a question in natural language suffers from ambiguity and insufficiently precise description of the goal. The problem is amplified when the system needs to generalize to databases unseen at training. In this paper, we consider the case when, at the test time, the system has access to an external criterion that evaluates the generated queries. The criterion can vary from checking that a query executes without errors to verifying the query on a set of tests. In this setting, we augment neural autoregressive models with a search algorithm that looks for a query satisfying the criterion. We apply our approach to the state-of-the-art semantic parsers and report that it allows us to find many queries passing all the tests on different datasets.