Dynamic Skill Adaptation for Large Language Models
This work addresses the challenge of efficiently teaching new skills to LLMs, which is incremental as it builds on existing data generation and instruction-tuning methods.
The authors tackled the problem of adapting complex skills to large language models by proposing Dynamic Skill Adaptation (DSA), which organizes training data based on human learning pathways and dynamically tailors it during training, resulting in effective adaptation of math reasoning and social study skills as demonstrated on models like LLAMA and Mistral.
We present Dynamic Skill Adaptation (DSA), an adaptive and dynamic framework to adapt novel and complex skills to Large Language Models (LLMs). Compared with previous work which learns from human-curated and static data in random orders, we propose to first automatically generate and organize the training data by mimicking the learning pathways of human and then dynamically tailor the training data based on the training dynamics. Specifically, inspired by the learning structures and teaching strategies in the human education system, we first construct a skill graph by decomposing complex skills into sub-skills and arranging them based on their dependencies in human syllables. For every skill, we utilize LLMs to generate both textbook-like data which contains detailed descriptions of skills for pre-training and exercise-like data which targets at explicitly utilizing the skills to solve problems for instruction-tuning. Furthermore, during the instruction-tuning, we dynamically update the training data which down-weight easy-to-learn examples, generate more complex examples, and filter out data with errors. Experiments on large language models such as LLAMA and Mistral demonstrate the effectiveness of our proposed methods in adapting math reasoning skills and social study skills.