Past Meets Present: Creating Historical Analogy with Large Language Models
This work addresses the difficulty in finding appropriate historical analogies for decision-making and understanding, a problem for historians, educators, and policymakers, but it is incremental as it builds on existing LLM techniques.
The paper tackles the problem of acquiring historical analogies for given events, which is challenging for people and overlooked in AI research, by exploring retrieval and generation methods using large language models (LLMs) and proposing a self-reflection method to reduce hallucinations and stereotypes. The results show that LLMs have good potential for this task, and performance improves with the self-reflection method, as validated through human evaluations and automatic multi-dimensional assessments.
Historical analogies, which compare known past events with contemporary but unfamiliar events, are important abilities that help people make decisions and understand the world. However, research in applied history suggests that people have difficulty finding appropriate analogies. And previous studies in the AI community have also overlooked historical analogies. To fill this gap, in this paper, we focus on the historical analogy acquisition task, which aims to acquire analogous historical events for a given event. We explore retrieval and generation methods for acquiring historical analogies based on different large language models (LLMs). Furthermore, we propose a self-reflection method to mitigate hallucinations and stereotypes when LLMs generate historical analogies. Through human evaluations and our specially designed automatic multi-dimensional assessment, we find that LLMs generally have a good potential for historical analogies. And the performance of the models can be further improved by using our self-reflection method.