Cerebra: Aligning Implicit Knowledge in Interactive SQL Authoring
This addresses the problem of unreliable SQL generation for users of LLM-driven tools, though it appears incremental as it builds on existing interactive approaches.
The paper tackles the problem of LLM-generated SQL queries being frequently erroneous due to underspecified user instructions that assume implicit knowledge, by presenting Cerebra, an interactive NL-to-SQL tool that aligns implicit knowledge between users and LLMs. In a user study with 16 participants, Cerebra demonstrated improved support for customized SQL authoring.
LLM-driven tools have significantly lowered barriers to writing SQL queries. However, user instructions are often underspecified, assuming the model understands implicit knowledge, such as dataset schemas, domain conventions, and task-specific requirements, that isn't explicitly provided. This results in frequently erroneous scripts that require users to repeatedly clarify their intent. Additionally, users struggle to validate generated scripts because they cannot verify whether the model correctly applied implicit knowledge. We present Cerebra, an interactive NL-to-SQL tool that aligns implicit knowledge between users and LLMs during SQL authoring. Cerebra automatically retrieves implicit knowledge from historical SQL scripts based on user instructions, presents this knowledge in an interactive tree view for code review, and supports iterative refinement to improve generated scripts. To evaluate the effectiveness and usability of Cerebra, we conducted a user study with 16 participants, demonstrating its improved support for customized SQL authoring. The source code of Cerebra is available at https://github.com/zjuidg/CHI26-Cerebra.