CEM: A Data-Efficient Method for Large Language Models to Continue Evolving From Mistakes
This addresses the problem of keeping LLMs current with evolving knowledge and tasks for AI practitioners, though it is incremental as it builds on existing continual learning strategies.
The paper tackles the challenge of efficiently updating large language models with new knowledge and tasks by proposing CEM, a data-efficient method that iteratively uses mistakes to guide continual pre-training and instruction tuning, resulting in performance gains of up to 29.63% on QA tasks.
As world knowledge advances and new task schemas emerge, Continual Learning (CL) becomes essential for keeping Large Language Models (LLMs) current and addressing their shortcomings. This process typically involves continual instruction tuning (CIT) and continual pre-training (CPT) to enable these models to adapt to novel tasks and acquire critical knowledge. However, collecting sufficient CPT data and efficiently bridging knowledge gaps remain significant challenges. Inspired by the 'summarizing mistakes' strategy, we propose the Continue Evolving from Mistakes (CEM) method, a data-efficient approach aiming to collect CPT data and continually improve LLMs' performance through iterative evaluation and supplementation with mistake-relevant knowledge. To further optimize data usage and mitigate forgetting, we introduce a novel training paradigm that combines CIT and CPT. Experiments show that CEM substantially enhances multiple models' performance on both in-domain and out-of-domain QA tasks, achieving gains of up to 29.63%. Code and datasets are available on https://anonymous.4open.science/r/cem-BB25.