Beyond Pixels: Semantic-aware Typographic Attack for Geo-Privacy Protection
This addresses a privacy threat for social media users from emerging LVLMs, offering a visually-preserving protection method that is incremental over existing adversarial perturbations.
The paper tackles the problem of protecting geo-privacy from Large Visual Language Models (LVLMs) that infer geolocation from images, by developing a semantics-aware typographical attack that adds deceptive text to images, significantly reducing prediction accuracy across five state-of-the-art LVLMs in experiments on three datasets.
Large Visual Language Models (LVLMs) now pose a serious yet overlooked privacy threat, as they can infer a social media user's geolocation directly from shared images, leading to unintended privacy leakage. While adversarial image perturbations provide a potential direction for geo-privacy protection, they require relatively strong distortions to be effective against LVLMs, which noticeably degrade visual quality and diminish an image's value for sharing. To overcome this limitation, we identify typographical attacks as a promising direction for protecting geo-privacy by adding text extension outside the visual content. We further investigate which textual semantics are effective in disrupting geolocation inference and design a two-stage, semantics-aware typographical attack that generates deceptive text to protect user privacy. Extensive experiments across three datasets demonstrate that our approach significantly reduces geolocation prediction accuracy of five state-of-the-art commercial LVLMs, establishing a practical and visually-preserving protection strategy against emerging geo-privacy threats.