Can AI Models be Jailbroken to Phish Elderly Victims? An End-to-End Evaluation
This work highlights critical AI safety failures that leave vulnerable elderly populations at risk of fraud, showing that current measures are insufficient.
The study demonstrated that attackers can jailbreak large language models to generate phishing content and successfully compromise elderly victims, with AI-generated phishing emails achieving an 11% success rate in a human validation study involving 108 senior volunteers.
We present an end-to-end demonstration of how attackers can exploit AI safety failures to harm vulnerable populations: from jailbreaking LLMs to generate phishing content, to deploying those messages against real targets, to successfully compromising elderly victims. We systematically evaluated safety guardrails across six frontier LLMs spanning four attack categories, revealing critical failures where several models exhibited near-complete susceptibility to certain attack vectors. In a human validation study with 108 senior volunteers, AI-generated phishing emails successfully compromised 11\% of participants. Our work uniquely demonstrates the complete attack pipeline targeting elderly populations, highlighting that current AI safety measures fail to protect those most vulnerable to fraud. Beyond generating phishing content, LLMs enable attackers to overcome language barriers and conduct multi-turn trust-building conversations at scale, fundamentally transforming fraud economics. While some providers report voluntary counter-abuse efforts, we argue these remain insufficient.