Minifinetuning: Low-Data Generation Domain Adaptation through Corrective Self-Distillation
This addresses the issue of overfitting in domain adaptation for language models when data is scarce, which is incremental as it builds on existing finetuning methods.
The paper tackles the problem of language model domain adaptation causing performance deterioration in low-data settings by introducing minifinetuning (MFT), which reduces overfitting-induced degeneralization without pre-training data, achieving 2-10x better specialization-to-degeneralization ratios than standard finetuning with as few as 500 samples.
Finetuning language models for a new domain inevitably leads to the deterioration of their general performance. This becomes more pronounced the more limited the finetuning data resource. We introduce minifinetuning (MFT), a method for language model domain adaptation that considerably reduces the effects of overfitting-induced degeneralization in low-data settings and which does so in the absence of any pre-training data for replay. MFT demonstrates 2-10x more favourable specialization-to-degeneralization ratios than standard finetuning across a wide range of models and domains and exhibits an intrinsic robustness to overfitting when data in the new domain is scarce and down to as little as 500 samples. Employing corrective self-distillation that is individualized on the sample level, MFT outperforms parameter-efficient finetuning methods, demonstrates replay-like degeneralization mitigation properties, and is composable with either for a combined effect.