Retrieval and Augmentation of Domain Knowledge for Text-to-SQL Semantic Parsing
This work addresses the challenge of domain-specific vocabulary in text-to-SQL parsing for database users, offering an incremental improvement over prior approaches.
The paper tackles the problem of translating natural language queries to SQL across diverse databases by proposing a systematic framework for retrieving and augmenting domain knowledge, demonstrating that DB-level structured domain statements and substring-based retrieval significantly improve accuracy over existing methods.
The performance of Large Language Models (LLMs) for translating Natural Language (NL) queries into SQL varies significantly across databases (DBs). NL queries are often expressed using a domain specific vocabulary, and mapping these to the correct SQL requires an understanding of the embedded domain expressions, their relationship to the DB schema structure. Existing benchmarks rely on unrealistic, ad-hoc query specific textual hints for expressing domain knowledge. In this paper, we propose a systematic framework for associating structured domain statements at the database level. We present retrieval of relevant structured domain statements given a user query using sub-string level match. We evaluate on eleven realistic DB schemas covering diverse domains across five open-source and proprietary LLMs and demonstrate that (1) DB level structured domain statements are more practical and accurate than existing ad-hoc query specific textual domain statements, and (2) Our sub-string match based retrieval of relevant domain statements provides significantly higher accuracy than other retrieval approaches.