Hey Alexa what did I just type? Decoding smartphone sounds with a voice assistant
This research highlights a significant privacy vulnerability for smartphone users, as always-on microphones in voice assistants can be used to infer typed input, extending keyboard-inference attacks to virtual keyboards.
This paper demonstrates that voice assistants can be exploited to extract sensitive data, such as PIN codes and text messages, typed on nearby smartphones. The attack was successful from recordings collected by a voice assistant up to half a meter away using two smartphones and a tablet.
Voice assistants are now ubiquitous and listen in on our everyday lives. Ever since they became commercially available, privacy advocates worried that the data they collect can be abused: might private conversations be extracted by third parties? In this paper we show that privacy threats go beyond spoken conversations and include sensitive data typed on nearby smartphones. Using two different smartphones and a tablet we demonstrate that the attacker can extract PIN codes and text messages from recordings collected by a voice assistant located up to half a meter away. This shows that remote keyboard-inference attacks are not limited to physical keyboards but extend to virtual keyboards too. As our homes become full of always-on microphones, we need to work through the implications.