Teaching According to Talents! Instruction Tuning LLMs with Competence-Aware Curriculum Learning
This work addresses the issue of sub-optimal learning trajectories in instruction tuning for LLMs, offering an incremental improvement over existing curriculum methods.
The paper tackles the problem of curriculum rigidity in instruction tuning for large language models by proposing CAMPUS, a framework that dynamically adjusts the curriculum based on model capabilities, resulting in superior performance compared to state-of-the-art baselines.
Efficient instruction tuning aims to enhance the ultimate performance of large language models (LLMs) trained on a given instruction dataset. Curriculum learning as a typical data organization strategy has shown preliminary effectiveness in instruction tuning. However, current curriculum tuning methods suffer from the curriculum rigidity, since they rely solely on static heuristic difficulty metrics. These methods fail to adapt to the evolving capabilities of models during training, resulting in a fixed and potentially sub-optimal learning trajectory. To address the issue, Competence-Aware Multi-Perspective cUrriculum inStruction tuning framework termed CAMPUS is proposed. CAMPUS offers several advantages: (1) Dynamic selection for sub-curriculum. (2) Competency-aware adjustment to the curriculum schedule. (3) Multiple difficulty-based scheduling. Extensive experiments prove the superior performance of CAMPUS, compared to other state-of-the-art baselines for efficient instruction tuning.