Small Language Models as Effective Guides for Large Language Models in Chinese Relation Extraction
This work addresses the long-tailed data issue in relation extraction for Chinese language processing, representing an incremental improvement in applying LLMs to this domain-specific challenge.
The paper tackles the long-tailed data problem in Chinese relation extraction by proposing SLCoLM, a framework that uses small language models to guide large language models, resulting in improved performance for long-tail relation types on an ancient Chinese dataset.
Recently, large language models (LLMs) have been successful in relational extraction (RE) tasks, especially in the few-shot learning. An important problem in the field of RE is long-tailed data, while not much attention is paid to this problem using LLM approaches. Therefore, in this paper, we propose SLCoLM, a model collaboration framework, to mitigate the data long-tail problem. In our framework, we use the ``\textit{Training-Guide-Predict}'' strategy to combine the strengths of small pre-trained language models (SLMs) and LLMs, where a task-specific SLM framework acts as a guider, transfers task knowledge to the LLM and guides the LLM in performing RE tasks. Our experiments on an ancient Chinese RE dataset rich in relation types show that the approach facilitates RE of long-tail relation types.