An In-Context Schema Understanding Method for Knowledge Base Question Answering
This addresses the problem of schema heterogeneity in knowledge base question answering for AI researchers, but it is incremental as it builds on existing in-context learning approaches.
The paper tackles the challenge of enabling Large Language Models (LLMs) to directly understand knowledge base schemas for question answering, proposing an In-Context Schema Understanding method that uses annotated examples, and it achieves competitive performance on KQA Pro and WebQSP datasets.
The Knowledge Base Question Answering (KBQA) task aims to answer natural language questions based on a given knowledge base. Recently, Large Language Models (LLMs) have shown strong capabilities in language understanding and can be used to solve this task. In doing so, a major challenge for LLMs is to overcome the immensity and heterogeneity of knowledge base schemas.Existing methods bypass this challenge by initially employing LLMs to generate drafts of logic forms without schema-specific details.Then, an extra module is used to inject schema information to these drafts.In contrast, in this paper, we propose a simple In-Context Schema Understanding (ICSU) method that enables LLMs to directly understand schemas by leveraging in-context learning. Specifically, ICSU provides schema information to LLMs using schema-related annotated examples. We investigate three example retrieval strategies based on raw questions, anonymized questions, and generated SPARQL queries. Experimental results show that ICSU demonstrates competitive performance compared to baseline methods on both the KQA Pro and WebQSP datasets.