Synesthesia: Detecting Screen Content via Remote Acoustic Side Channels
This reveals a novel privacy vulnerability for users of computers and video conferencing, enabling remote attackers to infer sensitive on-screen information without direct access.
The paper tackled the problem of detecting screen content via acoustic side channels, showing that subtle screen noises can be captured by common microphones to infer displayed text, virtual keyboard inputs, and web browsing activity in real-time, with attacks demonstrated from up to 10 meters away.
We show that subtle acoustic noises emanating from within computer screens can be used to detect the content displayed on the screens. This sound can be picked up by ordinary microphones built into webcams or screens, and is inadvertently transmitted to other parties, e.g., during a videoconference call or archived recordings. It can also be recorded by a smartphone or "smart speaker" placed on a desk next to the screen, or from as far as 10 meters away using a parabolic microphone. Empirically demonstrating various attack scenarios, we show how this channel can be used for real-time detection of on-screen text, or users' input into on-screen virtual keyboards. We also demonstrate how an attacker can analyze the audio received during video call (e.g., on Google Hangout) to infer whether the other side is browsing the web in lieu of watching the video call, and which web site is displayed on their screen.