An Actor-Critic Approach to Boosting Text-to-SQL Large Language Model
This addresses the lack of theoretical performance guarantees in existing empirical methods for Text-to-SQL conversion, offering a general enhancement approach for applications relying on natural language query interpretation.
The paper tackles the problem of improving Text-to-SQL conversion using large language models by proposing an Actor-Critic approach, where an Actor generates SQL queries and a Critic evaluates them iteratively, resulting in consistent performance improvements across multiple LLMs on datasets like Spider.
Text-To-SQL (T2S) conversion based on large language models (LLMs) has found a wide range of applications, by leveraging the capabilities of LLMs in interpreting the query intent expressed in natural language. Existing research focuses on suitable representations for data schema and/or questions, task-specific instructions and representative examples, and complicated inference pipelines. All these methods are empirical and task specific, without a theoretical bound on performance. In this paper, we propose a simple, general, and performance guaranteed T2S enhancement approach called Actor-Critic (AC). Specifically, we design two roles using the same LLM: an Actor to produce SQL queries and a Critic to evaluate the produced SQL. If the Critic believes the produced SQL is wrong, it notifies the Actor to reproduce the SQL and perform evaluation again. By this simple iterative process, expected performance can be derived in theory. We conducted extensive experiments on the Spider and related datasets with eleven LLMs, and demonstrated that the Actor-Critic method consistently improves the performance of T2S, thus serving as a general enhancement approach for T2S conversion.