BlendSQL: A Scalable Dialect for Unifying Hybrid Question Answering in Relational Algebra
This addresses the problem of inefficient and opaque reasoning in hybrid QA for users needing scalable and interpretable solutions, though it is incremental as it builds on existing SQL and QA frameworks.
The paper tackles the lack of control and scalability in hybrid question answering systems by introducing BlendSQL, a unified dialect based on SQLite that encodes reasoning steps into interpretable queries, resulting in a 35% reduction in token usage while improving performance on massive datasets.
Many existing end-to-end systems for hybrid question answering tasks can often be boiled down to a "prompt-and-pray" paradigm, where the user has limited control and insight into the intermediate reasoning steps used to achieve the final result. Additionally, due to the context size limitation of many transformer-based LLMs, it is often not reasonable to expect that the full structured and unstructured context will fit into a given prompt in a zero-shot setting, let alone a few-shot setting. We introduce BlendSQL, a superset of SQLite to act as a unified dialect for orchestrating reasoning across both unstructured and structured data. For hybrid question answering tasks involving multi-hop reasoning, we encode the full decomposed reasoning roadmap into a single interpretable BlendSQL query. Notably, we show that BlendSQL can scale to massive datasets and improve the performance of end-to-end systems while using 35% fewer tokens. Our code is available and installable as a package at https://github.com/parkervg/blendsql.