BadGPT-4o: stripping safety finetuning from GPT models
This work highlights a persistent vulnerability in AI safety for users and developers, as it demonstrates an easy-to-execute attack that compromises model safety without performance loss, though it is incremental as it builds on a known technique.
The researchers tackled the problem of removing safety guardrails from GPT-4o models by applying a known fine-tuning poisoning technique, resulting in an attack that matches best white-box jailbreaks on benchmarks like HarmBench and StrongREJECT without degrading performance or adding token overhead.
We show a version of Qi et al. 2023's simple fine-tuning poisoning technique strips GPT-4o's safety guardrails without degrading the model. The BadGPT attack matches best white-box jailbreaks on HarmBench and StrongREJECT. It suffers no token overhead or performance hits common to jailbreaks, as evaluated on tinyMMLU and open-ended generations. Despite having been known for a year, this attack remains easy to execute.