Are LLMs Overkill for Databases?: A Study on the Finiteness of SQL
This suggests that for database access, simpler template-based methods might be safer and cheaper than LLMs, addressing efficiency and reliability concerns in data retrieval.
The study analyzed 376 databases to show that SQL queries from natural language translations are finite in practical complexity, with 70% of queries covered by just 13% of template types, indicating high predictability.
Translating natural language to SQL for data retrieval has become more accessible thanks to code generation LLMs. But how hard is it to generate SQL code? While databases can become unbounded in complexity, the complexity of queries is bounded by real life utility and human needs. With a sample of 376 databases, we show that SQL queries, as translations of natural language questions are finite in practical complexity. There is no clear monotonic relationship between increases in database table count and increases in complexity of SQL queries. In their template forms, SQL queries follow a Power Law-like distribution of frequency where 70% of our tested queries can be covered with just 13% of all template types, indicating that the high majority of SQL queries are predictable. This suggests that while LLMs for code generation can be useful, in the domain of database access, they may be operating in a narrow, highly formulaic space where templates could be safer, cheaper, and auditable.