Research Challenges in Relational Database Management Systems for LLM Queries
This work addresses scalability and efficiency challenges for developers and companies integrating LLMs into database systems, but it is incremental as it builds on existing integrations.
The paper tackled the problem of limited functionality and poor performance in open-source relational database management systems for LLM queries, identifying issues such as enforcing structured outputs and optimizing resource utilization, and observed improvements through initial solutions.
Large language models (LLMs) have become essential for applications such as text summarization, sentiment analysis, and automated question-answering. Recently, LLMs have also been integrated into relational database management systems to enhance querying and support advanced data processing. Companies such as Amazon, Databricks, Google, and Snowflake offer LLM invocation directly within SQL, denoted as LLM queries, to boost data insights. However, open-source solutions currently have limited functionality and poor performance. In this work, we present an early exploration of two open-source systems and one enterprise platform, using five representative queries to expose functional, performance, and scalability limits in today's SQL-invoked LLM integrations. We identify three main issues: enforcing structured outputs, optimizing resource utilization, and improving query planning. We implemented initial solutions and observed improvements in accommodating LLM powered SQL queries. These early gains demonstrate that tighter integration of LLM+DBMS is the key to scalable and efficient processing of LLM queries.