Distillation Traps and Guards: A Calibration Knob for LLM Distillability
For practitioners deploying LLMs, this provides a practical knob to either enhance distillation or protect model IP, though the method is incremental over existing RFT techniques.
The paper identifies distillation traps that cause LLM knowledge distillation to fail and proposes a post-hoc calibration method using reinforcement fine-tuning to control teacher distillability. Experiments show calibrated teachers improve student performance or cause collapse for IP protection.
Knowledge distillation (KD) transfers capabilities from large language models (LLMs) to smaller students, yet it can fail unpredictably and also underpins model leakage risks. Our analysis revealed several distillation traps: tail noise, off-policy instability, and, most fundamentally, the teacher-student gap, that distort training signals. These traps manifest as overconfident hallucinations, self-correction collapse, and local decoding degradation, causing distillation to fail. Motivated by these findings, we propose a post-hoc calibration method that, to the best of our knowledge, for the first time enables control over a teacher's distillability via reinforcement fine-tuning (RFT). Our objective combines task utility, KL anchor, and across-tokenizer calibration reward. This makes distillability a practical safety lever for foundation models, connecting robust teacher-student transfer with deployment-aware model protection. Experiments across math, knowledge QA, and instruction-following tasks show that students distilled from distillable calibrated teachers outperform SFT and KD baselines, while undistillable calibrated teachers retain their task performance but cause distilled students to collapse, offering a practical knob for both better KD and model IP protection.