Enhancing the Capabilities of Large Language Models for API calls through Knowledge Graphs
It addresses the challenge of enhancing LLM capabilities for API calls in high-knowledge domains like meteorology, offering a novel solution that mitigates the need for costly fine-tuning and improves adaptability.
This paper tackles the problem of large language models (LLMs) having limited access to domain-specific knowledge for effective API calls in knowledge-intensive domains like meteorology, and introduces KG2data, a system that integrates knowledge graphs, LLMs, ReAct agents, and tool-use technologies to improve API call accuracy, achieving superior performance with metrics such as 1.43% name recognition failure, 0% hallucination failure, and 88.57% call correctness compared to baseline methods.
API calls by large language models (LLMs) offer a cutting-edge approach for data analysis. However, their ability to effectively utilize tools via API calls remains underexplored in knowledge-intensive domains like meteorology. This paper introduces KG2data, a system that integrates knowledge graphs, LLMs, ReAct agents, and tool-use technologies to enable intelligent data acquisition and query handling in the meteorological field. Using a virtual API, we evaluate API call accuracy across three metrics: name recognition failure, hallucination failure, and call correctness. KG2data achieves superior performance (1.43%, 0%, 88.57%) compared to RAG2data (16%, 10%, 72.14%) and chat2data (7.14%, 8.57%, 71.43%). KG2data differs from typical LLM-based systems by addressing their limited access to domain-specific knowledge, which hampers performance on complex or terminology-rich queries. By using a knowledge graph as persistent memory, our system enhances content retrieval, complex query handling, domain-specific reasoning, semantic relationship resolution, and heterogeneous data integration. It also mitigates the high cost of fine-tuning LLMs, making the system more adaptable to evolving domain knowledge and API structures. In summary, KG2data provides a novel solution for intelligent, knowledge-based question answering and data analysis in domains with high knowledge demands.