TableQA: a Large-Scale Chinese Text-to-SQL Dataset for Table-Aware SQL Generation
This addresses the gap in NL2SQL datasets for practical scenarios where users express values differently or query outside table knowledge, specifically for Chinese language applications.
The authors tackled the problem of natural language to SQL (NL2SQL) conversion by creating TableQA, a large-scale Chinese dataset with 64,891 questions and 20,311 SQL queries across 6,000 tables, which challenges existing models by requiring generalization to various value expressions and table-aware reasoning. They showed that state-of-the-art models drop from 95.1% to 46.8% condition value accuracy on this dataset, and their proposed table-aware approaches improved accuracy by 4.7% and 3.4%.
Parsing natural language to corresponding SQL (NL2SQL) with data driven approaches like deep neural networks attracts much attention in recent years. Existing NL2SQL datasets assume that condition values should appear exactly in natural language questions and the queries are answerable given the table. However, these assumptions may fail in practical scenarios, because user may use different expressions for the same content in the table, and query information outside the table without the full picture of contents in table. Therefore we present TableQA, a large-scale cross-domain Natural Language to SQL dataset in Chinese language consisting 64,891 questions and 20,311 unique SQL queries on over 6,000 tables. Different from exisiting NL2SQL datasets, TableQA requires to generalize well not only to SQL skeletons of different questions and table schemas, but also to the various expressions for condition values. Experiment results show that the state-of-the-art model with 95.1% condition value accuracy on WikiSQL only gets 46.8% condition value accuracy and 43.0% logic form accuracy on TableQA, indicating the proposed dataset is challenging and necessary to handle. Two table-aware approaches are proposed to alleviate the problem, the end-to-end approaches obtains 51.3% and 47.4% accuracy on the condition value and logic form tasks, with improvement of 4.7% and 3.4% respectively.