Joint Information Extraction Across Classical and Modern Chinese with Tea-MOELoRA
This addresses information extraction for Chinese documents across different eras, but it is incremental as it builds on existing LoRA and MoE methods.
The paper tackles the problem of Chinese information extraction across Classical and Modern documents by proposing Tea-MOELoRA, a parameter-efficient multi-task framework that combines LoRA with a Mixture-of-Experts design, resulting in outperformance of baselines.
Chinese information extraction (IE) involves multiple tasks across diverse temporal domains, including Classical and Modern documents. Fine-tuning a single model on heterogeneous tasks and across different eras may lead to interference and reduced performance. Therefore, in this paper, we propose Tea-MOELoRA, a parameter-efficient multi-task framework that combines LoRA with a Mixture-of-Experts (MoE) design. Multiple low-rank LoRA experts specialize in different IE tasks and eras, while a task-era-aware router mechanism dynamically allocates expert contributions. Experiments show that Tea-MOELoRA outperforms both single-task and joint LoRA baselines, demonstrating its ability to leverage task and temporal knowledge effectively.