Jenny Blessing

Nicholas Boucher, Jenny Blessing, Ilia Shumailov et al. · deepmind

Text-based machine learning models are vulnerable to an emerging class of Unicode-based adversarial examples capable of tricking a model into misreading text with potentially disastrous effects. The primary existing defense against these attacks is to preprocess potentially malicious text inputs using optical character recognition (OCR). In theory, OCR models will ignore any malicious Unicode characters and will extract the visually correct input to be fed to the model. In this work, we show that these visual defenses fail to prevent this type of attack. We use a genetic algorithm to generate visual adversarial examples (i.e., OCR outputs) in a black-box setting, demonstrating a highly effective novel attack that substantially reduces the accuracy of OCR and other visual models. Specifically, we use the Unicode functionality of combining characters (e.g., ñ which combines the characters n and ~) to manipulate text inputs so that small visual perturbations appear when the text is displayed. We demonstrate the effectiveness of these attacks in the real world by creating adversarial examples against production models published by Meta, Microsoft, IBM, and Google. We additionally conduct a user study to establish that the model-fooling adversarial examples do not affect human comprehension of the text, showing that language models are uniquely vulnerable to this type of text attack.

Jenny Blessing, Michael A. Specter, Daniel J. Weitzner

The security of the Internet rests on a small number of open-source cryptographic libraries: a vulnerability in any one of them threatens to compromise a significant percentage of web traffic. Despite this potential for security impact, the characteristics and causes of vulnerabilities in cryptographic software are not well understood. In this work, we conduct the first comprehensive analysis of cryptographic libraries and the vulnerabilities affecting them. We collect data from the National Vulnerability Database, individual project repositories and mailing lists, and other relevant sources for eight widely used cryptographic libraries. Among our most interesting findings is that only 27.2% of vulnerabilities in cryptographic libraries are cryptographic issues while 37.2% of vulnerabilities are memory safety issues, indicating that systems-level bugs are a greater security concern than the actual cryptographic procedures. In our investigation of the causes of these vulnerabilities, we find evidence of a strong correlation between the complexity of these libraries and their (in)security, empirically demonstrating the potential risks of bloated cryptographic codebases. We further compare our findings with non-cryptographic systems, observing that these systems are, indeed, more complex than similar counterparts, and that this excess complexity appears to produce significantly more vulnerabilities in cryptographic libraries than in non-cryptographic software.

Jenny Blessing, Julian Gomez, McCoy Patiño et al.

Voting by mail has been gaining traction for decades in the United States and has emerged as the preferred voting method during the COVID-19 pandemic. In this paper, we examine the security of electronic systems used in the process of voting by mail, including online voter registration and online ballot tracking systems. The goals of these systems, to facilitate voter registration and increase public confidence in elections, are laudable. They indisputably provide a critical public good. It is for these reasons that understanding the security and privacy posture of the mail-in voting process is paramount. We find that online voter registration systems in some states have vulnerabilities that allow adversaries to alter or effectively prevent a voter's registration. We additionally find that ballot tracking systems raise serious privacy questions surrounding ease of access to voter data. While the vulnerabilities discussed here are unlikely to enable an adversary to modify votes, several could have the effect of disenfranchising voters and reducing voter confidence in U.S. elections infrastructure, thereby undermining the very purpose of these systems.