Make a Choice! Knowledge Base Question Answering with In-Context Learning
This work addresses the problem of limited annotated data for KBQA, offering a method to enhance generalization for researchers and practitioners in NLP, though it is incremental as it builds on existing LLM and KBQA techniques.
The paper tackles the challenge of generalizing knowledge base question answering (KBQA) in low-resource settings by incorporating large language models (LLMs) with in-context learning, resulting in competitive performance and strong improvements in generalization on two KBQA datasets.
Question answering over knowledge bases (KBQA) aims to answer factoid questions with a given knowledge base (KB). Due to the large scale of KB, annotated data is impossible to cover all fact schemas in KB, which poses a challenge to the generalization ability of methods that require a sufficient amount of annotated data. Recently, LLMs have shown strong few-shot performance in many NLP tasks. We expect LLM can help existing methods improve their generalization ability, especially in low-resource situations. In this paper, we present McL-KBQA, a framework that incorporates the few-shot ability of LLM into the KBQA method via ICL-based multiple choice and then improves the effectiveness of the QA tasks. Experimental results on two KBQA datasets demonstrate the competitive performance of McL-KBQA with strong improvements in generalization. We expect to explore a new way to QA tasks from KBQA in conjunction with LLM, how to generate answers normatively and correctly with strong generalization.