What Your Posts Reveal: A Benchmark and Agentic Framework for User-Level Privacy Leakage on Social Media
For researchers and practitioners concerned with privacy leakage on social media, this work provides a benchmark and metric to evaluate leakage severity, with a novel agentic framework that outperforms existing methods.
The paper addresses the lack of a unified benchmark and evaluation metric for user-level multimodal privacy leakage on social media. It introduces SopriBench, a synthetic benchmark with 50 user profiles and 1,569 images, and the Privacy Exposure Score (PES) metric. The proposed agentic framework Argus achieves a PES of 0.55, a 25% improvement over the strongest baseline.
Public social media posts can reveal private information through weak cues scattered across text, images, or metadata. Such leakage is often cumulative and cross-post: cues that appear harmless in isolation may jointly expose a user's home, workplace, or routine. However, current research lacks a unified benchmark for user-level multimodal privacy leakage and an evaluation metric that captures exposure severity beyond binary accuracy. To address these gaps, we propose SopriBench, a synthetic benchmark guided by leakage patterns abstracted from a private reference corpus of Rednote and Instagram accounts, covering 50 user profiles and 1,569 images with attributes, contextual sensitivity, granularity, leakage type, inference difficulty, and supporting evidence. We further introduce the Privacy Exposure Score (PES), which weights value granularity by contextual sensitivity. Inspired by abductive reasoning, we introduce Argus, a training-free agentic framework for cumulative leakage inference. Argus forms hypotheses from accumulated evidence, verifies supporting evidence, and aggregates cross-post cues into privacy profiles, achieving 0.55 PES, a 25% improvement over the strongest baseline, with the largest gain on cross-post leakage.