Invisible Safety Threat: Malicious Finetuning for LLM via Steganography
This research highlights a critical, previously unaddressed safety alignment risk for LLM developers and users, demonstrating how malicious content can bypass existing safeguards and remain undetected.
The paper identifies an invisible safety threat in LLMs where a finetuned model can covertly generate harmful content using steganography, while appearing benign to human observers. This attack was demonstrated on GPT-4.1, Llama-3.3-70B-Instruct, Phi-4, and Mistral-Small-24B-Base-2501, with all steganographically embedded malicious content being misclassified as safe by Llama-Guard-3-8B.
Understanding and addressing potential safety alignment risks in large language models (LLMs) is critical for ensuring their safe and trustworthy deployment. In this paper, we highlight an insidious safety threat: a compromised LLM can maintain a facade of proper safety alignment while covertly generating harmful content. To achieve this, we finetune the model to understand and apply a steganographic technique. At inference time, we input a prompt that contains a steganographically embedded malicious target question along with a plaintext cover question. The model, in turn, produces a target response similarly embedded within a benign-looking cover response. In this process, human observers only see the model being prompted with a cover question and generating a corresponding cover response, while the malicious content is hidden from view. We demonstrate this invisible safety threat on GPT-4.1 despite the OpenAI finetuning API's safeguards. The finetuned model produces steganographic malicious outputs in response to hidden malicious prompts, while the user interface displays only a fully benign cover interaction. We also replicate the attack on three open-source models, Llama-3.3-70B-Instruct, Phi-4, and Mistral-Small-24B-Base-2501, confirming the generality of our method. We quantitatively evaluate our method on the AdvBench dataset, using Llama-Guard-3-8B for content safety classification. Across all four models, all stegotexts containing malicious content are incorrectly classified as safe.