ANALOGYKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Base
This addresses the lack of training resources for analogical reasoning in language models, offering a scalable solution with potential broad impact in AI, though it is incremental as it builds on existing knowledge graphs and LLMs.
The authors tackled the problem of language models struggling with analogical reasoning by creating ANALOGYKB, a million-scale analogy knowledge base derived from knowledge graphs, which improved performance on analogy recognition and generation tasks for both smaller and large language models.
Analogical reasoning is a fundamental cognitive ability of humans. However, current language models (LMs) still struggle to achieve human-like performance in analogical reasoning tasks due to a lack of resources for model training. In this work, we address this gap by proposing ANALOGYKB, a million-scale analogy knowledge base (KB) derived from existing knowledge graphs (KGs). ANALOGYKB identifies two types of analogies from the KGs: 1) analogies of the same relations, which can be directly extracted from the KGs, and 2) analogies of analogous relations, which are identified with a selection and filtering pipeline enabled by large language models (LLMs), followed by minor human efforts for data quality control. Evaluations on a series of datasets of two analogical reasoning tasks (analogy recognition and generation) demonstrate that ANALOGYKB successfully enables both smaller LMs and LLMs to gain better analogical reasoning capabilities.