ColloQL: Robust Cross-Domain Text-to-SQL Over Search Queries
This work addresses the need for robust text-to-SQL systems for non-technical users dealing with noisy, search-like queries, representing a domain-specific advancement.
The paper tackles the problem of translating colloquial and noisy natural language search queries into SQL, which is challenging due to the lack of suitable datasets and the informal nature of real-world user inputs. They introduce ColloQL, a model that achieves 84.9% logical and 90.7% execution accuracy on WikiSQL, setting a new state-of-the-art without execution-guided decoding.
Translating natural language utterances to executable queries is a helpful technique in making the vast amount of data stored in relational databases accessible to a wider range of non-tech-savvy end users. Prior work in this area has largely focused on textual input that is linguistically correct and semantically unambiguous. However, real-world user queries are often succinct, colloquial, and noisy, resembling the input of a search engine. In this work, we introduce data augmentation techniques and a sampling-based content-aware BERT model (ColloQL) to achieve robust text-to-SQL modeling over natural language search (NLS) questions. Due to the lack of evaluation data, we curate a new dataset of NLS questions and demonstrate the efficacy of our approach. ColloQL's superior performance extends to well-formed text, achieving 84.9% (logical) and 90.7% (execution) accuracy on the WikiSQL dataset, making it, to the best of our knowledge, the highest performing model that does not use execution guided decoding.