WenyanGPT: A Large Language Model for Classical Chinese Tasks
This work addresses the need for better language processing tools for Classical Chinese, which is crucial for cultural heritage and academic study, but it is incremental as it builds on existing models like LLaMA3-8B-Chinese.
The paper tackles the problem of inadequate performance of existing natural language processing models on Classical Chinese by developing WenyanGPT, a large language model specifically designed for Classical Chinese tasks, which significantly outperforms current advanced LLMs on the WenyanBENCH evaluation benchmark.
Classical Chinese, as the core carrier of Chinese culture, plays a crucial role in the inheritance and study of ancient literature. However, existing natural language processing models primarily optimize for Modern Chinese, resulting in inadequate performance on Classical Chinese. This paper presents a comprehensive solution for Classical Chinese language processing. By continuing pre-training and instruction fine-tuning on the LLaMA3-8B-Chinese model, we construct a large language model, WenyanGPT, which is specifically designed for Classical Chinese tasks. Additionally, we develop an evaluation benchmark dataset, WenyanBENCH. Experimental results on WenyanBENCH demonstrate that WenyanGPT significantly outperforms current advanced LLMs in various Classical Chinese tasks. We make the model's training data, instruction fine-tuning data\footnote, and evaluation benchmark dataset publicly available to promote further research and development in the field of Classical Chinese processing.