Kongzi: A Historical Large Language Model with Fact Enhancement
This addresses the need for accurate and reliable LLMs in professional domains like historical studies, though it is incremental as it builds on existing LLM capabilities with domain-specific enhancements.
The paper tackles the problem of factual inaccuracies in large language models during complex reasoning tasks, particularly in historical analysis, by proposing Kongzi, a model that integrates curated historical data and a fact-reinforcement learning strategy, resulting in outperformance in factual accuracy and reasoning depth in experiments.
The capabilities of the latest large language models (LLMs) have been extended from pure natural language understanding to complex reasoning tasks. However, current reasoning models often exhibit factual inaccuracies in longer reasoning chains, which poses challenges for historical reasoning and limits the potential of LLMs in complex, knowledge-intensive tasks. Historical studies require not only the accurate presentation of factual information but also the ability to establish cross-temporal correlations and derive coherent conclusions from fragmentary and often ambiguous sources. To address these challenges, we propose Kongzi, a large language model specifically designed for historical analysis. Through the integration of curated, high-quality historical data and a novel fact-reinforcement learning strategy, Kongzi demonstrates strong factual alignment and sophisticated reasoning depth. Extensive experiments on tasks such as historical question answering and narrative generation demonstrate that Kongzi outperforms existing models in both factual accuracy and reasoning depth. By effectively addressing the unique challenges inherent in historical texts, Kongzi sets a new standard for the development of accurate and reliable LLMs in professional domains.