Mitigating Catastrophic Forgetting in Mathematical Reasoning Finetuning through Mixed Training
This addresses the problem of catastrophic forgetting for researchers and practitioners finetuning models on specialized tasks, offering a practical solution to preserve general capabilities, though it is incremental in applying existing regularization techniques.
The paper tackled catastrophic forgetting in large language models during finetuning for mathematical reasoning, showing that math-only training improved accuracy from 3.1% to 12.0% but caused NLI accuracy to drop from 81.0% to 16.5%. They proposed mixed training strategies, which eliminated forgetting while maintaining math performance, achieving 12.0% math accuracy and 86.2% NLI accuracy with a 1:1 ratio.
When finetuning large language models for specialized tasks such as mathematical reasoning, models exhibit catastrophic forgetting, losing previously learned capabilities. We investigate this by finetuning Flan-T5-Base (250M parameters) on the DeepMind Mathematics dataset and measuring forgetting on MultiNLI. Math-only training improves mathematical accuracy from 3.1\% to 12.0\% but causes NLI accuracy to collapse from 81.0\% to 16.5\%--a 64.5 percentage point drop occurring within the first 1,000 training steps. We propose mixed training strategies that interleave mathematical and NLI examples during training. Our results demonstrate that mixed training completely eliminates catastrophic forgetting while maintaining equivalent mathematical performance: the balanced 1:1 ratio achieves 12.0\% math accuracy (matching math-only) while preserving 86.2\% NLI accuracy. We systematically explore mixing ratios from 1:1 to 15:1, finding that even minimal NLI exposure (6.2\%) provides effective regularization. These findings demonstrate that specialization need not require forgetting general capabilities, with implications for scaling to larger models where mixed training may confer additional benefits beyond forgetting prevention.