The TIP of the Iceberg: Revealing a Hidden Class of Task-in-Prompt Adversarial Attacks on LLMs
This reveals critical weaknesses in LLM safety alignments, posing a security problem for AI developers and users, and is a novel attack method rather than incremental.
The paper tackles the problem of jailbreaking LLMs by introducing Task-in-Prompt (TIP) attacks that embed tasks like cipher decoding into prompts to generate prohibited inputs, and it demonstrates these attacks successfully circumvent safeguards in six state-of-the-art models including GPT-4o and LLaMA 3.2.
We present a novel class of jailbreak adversarial attacks on LLMs, termed Task-in-Prompt (TIP) attacks. Our approach embeds sequence-to-sequence tasks (e.g., cipher decoding, riddles, code execution) into the model's prompt to indirectly generate prohibited inputs. To systematically assess the effectiveness of these attacks, we introduce the PHRYGE benchmark. We demonstrate that our techniques successfully circumvent safeguards in six state-of-the-art language models, including GPT-4o and LLaMA 3.2. Our findings highlight critical weaknesses in current LLM safety alignments and underscore the urgent need for more sophisticated defence strategies. Warning: this paper contains examples of unethical inquiries used solely for research purposes.