Aligning Large Language Models to a Domain-specific Graph Database for NL2GQL
This addresses the problem of querying domain-specific graph databases for users in fields like finance and medicine, but it is incremental as it adapts existing LLM methods to a new task.
The paper tackles the challenge of translating natural language to graph query language (NL2GQL) for domain-specific graph databases by aligning large language models (LLMs) using a pipeline that generates NL-GQL data pairs with ChatGPT and fine-tunes LLMs, resulting in improvements of 5.90-6.36 EM and 6.00-7.09 EX points over baselines on finance and medicine datasets.
Graph Databases (Graph DB) find extensive application across diverse domains such as finance, social networks, and medicine. Yet, the translation of Natural Language (NL) into the Graph Query Language (GQL), referred to as NL2GQL, poses significant challenges owing to its intricate and specialized nature. Some approaches have sought to utilize Large Language Models (LLMs) to address analogous tasks like text2SQL. Nonetheless, in the realm of NL2GQL tasks tailored to a particular domain, the absence of domain-specific NL-GQL data pairs adds complexity to aligning LLMs with the graph DB. To tackle this challenge, we present a well-defined pipeline. Initially, we utilize ChatGPT to generate NL-GQL data pairs, leveraging the provided graph DB with self-instruction. Subsequently, we employ the generated data to fine-tune LLMs, ensuring alignment between LLMs and the graph DB. Moreover, we find the importance of relevant schema in efficiently generating accurate GQLs. Thus, we introduce a method to extract relevant schema as the input context. We evaluate our method using two carefully constructed datasets derived from graph DBs in the finance and medicine domains, named FinGQL and MediGQL. Experimental results reveal that our approach significantly outperforms a set of baseline methods, with improvements of 5.90 and 6.36 absolute points on EM, and 6.00 and 7.09 absolute points on EX for FinGQL and MediGQL, respectively.