Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbation
This addresses safety concerns in fine-tuning-as-a-service for large language models, offering an incremental improvement over existing defenses.
The paper tackles the problem of harmful fine-tuning attacks on large language models by identifying harmful weight perturbations as a cause of alignment breakdown, and proposes Booster, an alignment-stage solution that reduces harmful scores by up to 40% while maintaining downstream task performance.
Harmful fine-tuning attack poses serious safety concerns for large language models' fine-tuning-as-a-service. While existing defenses have been proposed to mitigate the issue, their performances are still far away from satisfactory, and the root cause of the problem has not been fully recovered. To this end, we in this paper show that harmful perturbation over the model weights could be a probable cause of alignment-broken. In order to attenuate the negative impact of harmful perturbation, we propose an alignment-stage solution, dubbed Booster. Technically, along with the original alignment loss, we append a loss regularizer in the alignment stage's optimization. The regularizer ensures that the model's harmful loss reduction after the simulated harmful perturbation is attenuated, thereby mitigating the subsequent fine-tuning risk. Empirical results show that Booster can effectively reduce the harmful score of the fine-tuned models while maintaining the performance of downstream tasks. Our code is available at https://github.com/git-disl/Booster.