SelECT-SQL: Self-correcting ensemble Chain-of-Thought for Text-to-SQL
This addresses the challenge of achieving expert-level performance in Text-to-SQL for natural language processing and data management, with incremental improvements over existing methods.
The paper tackled the problem of improving Text-to-SQL conversion accuracy by introducing SelECT-SQL, a method combining chain-of-thought prompting, self-correction, and ensemble techniques, achieving 84.2% execution accuracy on the Spider benchmark, surpassing previous GPT-3.5-Turbo and GPT-4 results.
In recent years,Text-to-SQL, the problem of automatically converting questions posed in natural language to formal SQL queries, has emerged as an important problem at the intersection of natural language processing and data management research. Large language models (LLMs) have delivered impressive performance when used in an off-the-shelf performance, but still fall significantly short of expected expert-level performance. Errors are especially probable when a nuanced understanding is needed of database schemas, questions, and SQL clauses to do proper Text-to-SQL conversion. We introduce SelECT-SQL, a novel in-context learning solution that uses an algorithmic combination of chain-of-thought (CoT) prompting, self-correction, and ensemble methods to yield a new state-of-the-art result on challenging Text-to-SQL benchmarks. Specifically, when configured using GPT-3.5-Turbo as the base LLM, SelECT-SQL achieves 84.2% execution accuracy on the Spider leaderboard's development set, exceeding both the best results of other baseline GPT-3.5-Turbo-based solutions (81.1%), and the peak performance (83.5%) of the GPT-4 result reported on the leaderboard.