Automated Profile Inference with Language Model Agents
For users of pseudonymous online platforms, this work highlights a novel privacy threat from LLM agents that can automatically infer sensitive personal attributes, raising urgent concerns for privacy protection.
This paper introduces automated profile inference, a new privacy threat where LLM agents can extract sensitive personal attributes from pseudonymous online activities. The proposed AutoProfiler framework achieves high effectiveness and efficiency across real-world and synthetic datasets, posing significant privacy risks.
Impressive progress has been made in automated problem-solving by the collaboration of large language model (LLM) based agents. However, these automated capabilities also open avenues for malicious applications. In this paper, we study a new threat that LLMs pose to online pseudonymity, called automated profile inference, where an adversary can instruct LLMs to automatically collect and extract sensitive personal attributes from publicly available user activities on pseudonymous platforms. We also introduce an automated profiling framework called AutoProfiler to demonstrate and assess the feasibility of such attacks in real-world scenarios. AutoProfiler consists of four specialized LLM agents that work collaboratively to retrieve and process user online activities and generate a profile with extracted personal information. Experimental results on two real-world datasets and one synthetic dataset show that AutoProfiler is highly effective and efficient, and the inferred attributes are both identifiable and sensitive, posing significant privacy risks. We explore mitigation strategies from different perspectives and advocate for increased public awareness of this emerging privacy threat.