Sockpuppetting: Jailbreaking LLMs Without Optimization Through Output Prefix Injection
This is an incremental improvement for security researchers and developers, as it provides a low-cost attack method accessible to unsophisticated adversaries, highlighting vulnerabilities in open-weight models.
The paper tackled the problem of jailbreaking open-weight large language models (LLMs) by introducing 'sockpuppetting', a simple method that injects an acceptance sequence at the start of the model's output, achieving up to 80% higher attack success rate than GCG on Qwen3-8B and 64% higher on Llama-3.1-8B with a hybrid approach.
As open-weight large language models (LLMs) increase in capabilities, safeguarding them against malicious prompts and understanding possible attack vectors becomes ever more important. While automated jailbreaking methods like GCG [Zou et al., 2023] remain effective, they often require substantial computational resources and specific expertise. We introduce "sockpuppetting'', a simple method for jailbreaking open-weight LLMs by inserting an acceptance sequence (e.g., "Sure, here is how to...'') at the start of a model's output and allowing it to complete the response. Requiring only a single line of code and no optimization, sockpuppetting achieves up to 80% higher attack success rate (ASR) than GCG on Qwen3-8B in per-prompt comparisons. We also explore a hybrid approach that optimizes the adversarial suffix within the assistant message block rather than the user prompt, increasing ASR by 64% over GCG on Llama-3.1-8B in a prompt-agnostic setting. The results establish sockpuppetting as an effective low-cost attack accessible to unsophisticated adversaries, highlighting the need for defences against output-prefix injection in open-weight models.