Tool-Assisted Agent on SQL Inspection and Refinement in Real-World Scenarios
This addresses a specific issue in real-world SQL query refinement for database management, but it is incremental as it builds on existing LLM-based methods with specialized tools.
The paper tackles the problem of database mismatches in Text-to-SQL systems, which are errors not caught by execution exceptions, and proposes a tool-assisted agent framework that achieves the highest performance on averaged results of Spider and Spider-Realistic datasets in few-shot settings and significantly outperforms baselines on the new Spider-Mismatch dataset.
Recent Text-to-SQL methods leverage large language models (LLMs) by incorporating feedback from the database management system. While these methods effectively address execution errors in SQL queries, they struggle with database mismatches -- errors that do not trigger execution exceptions. Database mismatches include issues such as condition mismatches and stricter constraint mismatches, both of which are more prevalent in real-world scenarios. To address these challenges, we propose a tool-assisted agent framework for SQL inspection and refinement, equipping the LLM-based agent with two specialized tools: a retriever and a detector, designed to diagnose and correct SQL queries with database mismatches. These tools enhance the capability of LLMs to handle real-world queries more effectively. We also introduce Spider-Mismatch, a new dataset specifically constructed to reflect the condition mismatch problems encountered in real-world scenarios. Experimental results demonstrate that our method achieves the highest performance on the averaged results of the Spider and Spider-Realistic datasets in few-shot settings, and it significantly outperforms baseline methods on the more realistic dataset, Spider-Mismatch.