iPDB -- Optimizing Semantic SQL Queries
This addresses the problem of inefficient data migration and complex engineering for developers needing to integrate learned models with SQL databases, though it is incremental as it builds on existing relational systems.
The paper tackles the incompatibility between SQL databases and machine learning model inference by introducing iPDB, a relational system that supports in-database ML and LLM inferencing with extended SQL syntax, achieving a 2.5x mean speedup and up to 30x speedup over state-of-the-art systems.
Structured Query Language (SQL) has remained the standard query language for databases. SQL is highly optimized for processing structured data laid out in relations. Meanwhile, in the present application development landscape, it is highly desirable to utilize the power of learned models to perform complex tasks. Large language models (LLMs) have been shown to understand and extract information from unstructured textual data. However, SQL as a query language and accompanying relational database systems are either incompatible or inefficient for workloads that require leveraging learned models. This results in complex engineering and multiple data migration operations that move data between the data sources and the model inference platform. In this paper, we present iPDB, a relational system that supports in-database machine learning (ML) and large language model (LLM) inferencing using extended SQL syntax. In iPDB, LLMs and ML calls can function as semantic projects, as predicates to perform semantic selects and semantic joins, or for semantic aggregations in group-by clauses. iPDB has a new relational predict operator along with semantic query optimizations that enable users to write and efficiently execute semantic SQL queries, outperforming other state-of-the-art systems by 2.5x mean speedup, with speedups of up to 30x.