Prompts have evil twins
This reveals a vulnerability in language models where prompts can be obfuscated while maintaining functionality, which is incremental but highlights security concerns.
The paper discovered that natural-language prompts can be replaced by 'evil twins'—unintelligible but functionally similar prompts—that transfer across language models, solving a maximum-likelihood problem to find them.
We discover that many natural-language prompts can be replaced by corresponding prompts that are unintelligible to humans but that provably elicit similar behavior in language models. We call these prompts "evil twins" because they are obfuscated and uninterpretable (evil), but at the same time mimic the functionality of the original natural-language prompts (twins). Remarkably, evil twins transfer between models. We find these prompts by solving a maximum-likelihood problem which has applications of independent interest.