Few-shot Transfer Learning for Knowledge Base Question Answering: Fusing Supervised Models with In-Context Learning
This addresses the high annotation cost for deploying KBQA systems in new domains, offering a practical solution for few-shot scenarios.
The paper tackles the problem of few-shot transfer learning for Knowledge Base Question Answering (KBQA), where target domains have limited labeled data, by proposing FuSIC-KBQA, which fuses supervised models with in-context learning, and results show it significantly outperforms state-of-the-art KBQA models in both transfer and limited-data in-domain settings.
Existing Knowledge Base Question Answering (KBQA) architectures are hungry for annotated data, which make them costly and time-consuming to deploy. We introduce the problem of few-shot transfer learning for KBQA, where the target domain offers only a few labeled examples, but a large labeled training dataset is available in a source domain. We propose a novel KBQA architecture called FuSIC-KBQA that performs KB-retrieval using multiple source-trained retrievers, re-ranks using an LLM and uses this as input for LLM few-shot in-context learning to generate logical forms. These are further refined using execution-guided feedback. Experiments over multiple source-target KBQA pairs of varying complexity show that FuSIC-KBQA significantly outperforms adaptations of SoTA KBQA models for this setting. Additional experiments show that FuSIC-KBQA also outperforms SoTA KBQA models in the in-domain setting when training data is limited.