Valid Text-to-SQL Generation with Unification-based DeepStochLog
This addresses the limitation of unreliable text-to-SQL systems for real-life applications, representing an incremental improvement by integrating existing methods.
The paper tackles the problem of generating invalid SQL queries from natural language using large language models by proposing a neurosymbolic framework that enforces SQL syntax and schema constraints, resulting in all output queries being valid and improving validity, execution accuracy, and alignment with ground truth.
Large language models have been used to translate natural language questions to SQL queries. Without hard constraints on syntax and database schema, they occasionally produce invalid queries that are not executable. These failures limit the usage of these systems in real-life scenarios. We propose a neurosymbolic framework that imposes SQL syntax and schema constraints with unification-based definite clause grammars and thus guarantees the generation of valid queries. Our framework also builds a bi-directional interface to language models to leverage their natural language understanding abilities. The evaluation results on a subset of SQL grammars show that all our output queries are valid. This work is the first step towards extending language models with unification-based grammars. We demonstrate this extension enhances the validity, execution accuracy, and ground truth alignment of the underlying language model by a large margin. Our code is available at https://github.com/ML-KULeuven/deepstochlog-lm.