Scott Ruoti

Abubakar Sadiq Shittu, Clay Shubert, John Sadik et al.

Mutual TLS (mTLS) provides strong, certificate-based authentication for both clients and servers, yet its adoption for user-facing websites remains rare. This paper presents a longitudinal study of mTLS usability, tracking 46 senior and graduate computer science students who configured client certificates from scratch, used them for routine authentication over a semester-long course, and managed credentials across multiple devices. The results reveal that initial setup is a major bottleneck; while daily use was considered smooth, it did not improve long-term usability perceptions. Most concerningly, only 9% of participants fully understood the security implications of certificate-based authentication. We conclude that in a realistic, tooling-heavy deployment utilizing OpenSSL, a custom CA, and a 3072-bit minimum key requirement, even highly technical students struggled significantly. We argue this provides empirical evidence that today mTLS user experience is fundamentally misaligned with non-PKI specialists, and it is difficult to see a path toward mainstream adoption without substantial platform-level changes.

Abubakar Sadiq Shittu, John Sadik, Farzin Gholamrezae et al.

Commit signing is widely promoted as a foundation of software supply-chain security, yet prior work has studied it through the lens of individual repositories or curated project samples, missing the broader picture of how developers behave across an entire platform. Grounded in replicability theory, we vary the sampling unit from repositories to individual developers, following 71,694 active GitHub users, defined as accounts that have authored at least one commit, across all their repositories and their entire commit history, spanning 16 million commits and 874,198 repositories. This platform-wide, user-centric view reveals a fundamental gap that repository sampling cannot detect. The ecosystem's apparent high signing adoption rate is an illusion. Once platform-generated signatures are excluded, fewer than 6% of developers have ever signed a commit themselves, and the vast majority of apparent signers have never signed outside a web browser. Among the minority who do sign locally, signing rarely persists over time or across repositories, and roughly one in eight developer-managed signatures fails verification because signing keys are never uploaded to GitHub. Examining the key registry, we find that expired keys are almost never revoked and more than a quarter of users carry at least one dead key. Together, these findings reveal that commit signing as practiced today cannot serve as a dependable provenance signal at ecosystem scale, and we offer concrete recommendations for closing that gap.

Ran Elgedawy, Porter Dosch, John Sadik et al.

$ $Large Language Models (LLMs) are being increasingly utilized in various applications, with code generations being a notable example. While previous research has shown that LLMs have the capability to generate both secure and insecure code, the literature does not take into account what factors help generate secure and effective code. Therefore in this paper we focus on identifying and understanding the conditions and contexts in which LLMs can be effectively and safely deployed in real-world scenarios to generate quality code. We conducted a comparative analysis of four advanced LLMs--GPT-3.5 and GPT-4 using ChatGPT and Bard and Gemini from Google--using 9 separate tasks to assess each model's code generation capabilities. We contextualized our study to represent the typical use cases of a real-life developer employing LLMs for everyday tasks as work. Additionally, we place an emphasis on security awareness which is represented through the use of two distinct versions of our developer persona. In total, we collected 61 code outputs and analyzed them across several aspects: functionality, security, performance, complexity, and reliability. These insights are crucial for understanding the models' capabilities and limitations, guiding future development and practical applications in the field of automated code generation.

Sean Oesch, Anuj Gautam, Scott Ruoti

Password managers help users more effectively manage their passwords, encouraging them to adopt stronger passwords across their many accounts. In contrast to desktop systems where password managers receive no system-level support, mobile operating systems provide autofill frameworks designed to integrate with password managers to provide secure and usable autofill for browsers and other apps installed on mobile devices. In this paper, we evaluate mobile autofill frameworks on iOS and Android, examining whether they achieve substantive benefits over the ad-hoc desktop environment or become a problematic single point of failure. Our results find that while the frameworks address several common issues, they also enforce insecure behavior and fail to provide password managers sufficient information to override the frameworks' insecure behavior, resulting in mobile managers being less secure than their desktop counterparts overall. We also demonstrate how these frameworks act as a confused deputy in manager-assisted credential phishing attacks. Our results demonstrate the need for significant improvements to mobile autofill frameworks. We conclude the paper with recommendations for the design and implementation of secure autofill frameworks.

Scott Ruoti, Ben Kaiser, Arkady Yerukhimovich et al.

Bitcoin's success has led to significant interest in its underlying components, particularly Blockchain technology. Over 10 years after Bitcoin's initial release, the community still suffers from a lack of clarity regarding what properties defines Blockchain technology, its relationship to similar technologies, and which of its proposed use-cases are tenable and which are little more than hype. In this paper we answer four common questions regarding Blockchain technology: (1) what exactly is Blockchain technology, (2) what capabilities does it provide, and (3) what are good applications for Blockchain technology, and (4) how does it relate to other approache distributed technologies (e.g., distributed databases). We accomplish this goal by using grounded theory (a structured approach to gathering and analyzing qualitative data) to thoroughly analyze a large corpus of literature on Blockchain technology. This method enables us to answer the above questions while limiting researcher bias, separating thought leadership from peddled hype and identifying open research questions related to Blockchain technology. The audience for this paper is broad as it aims to help researchers in a variety of areas come to a better understanding of Blockchain technology and identify whether it may be of use in their own research.

Sean Oesch, Scott Ruoti

Password managers have the potential to help users more effectively manage their passwords and address many of the concerns surrounding password-based authentication, however prior research has identified significant vulnerabilities in existing password managers. Since that time, five years has passed, leaving it unclear whether password managers remain vulnerable or whether they are now ready for broad adoption. To answer this question, we evaluate thirteen popular password managers and consider all three stages of the password manager lifecycle--password generation, storage, and autofill. Our evaluation is the first analysis of password generation in password managers, finding several non-random character distributions and identifying instances where generated passwords were vulnerable to online and offline guessing attacks. For password storage and autofill, we replicate past evaluations, demonstrating that while password managers have improved in the half-decade since those prior evaluations, there are still significant issues, particularly with browser-based password managers; these problems include unencrypted metadata, unsafe defaults, and vulnerabilities to clickjacking attacks. Based on our results, we identify password managers to avoid, provide recommendations on how to improve existing password managers, and identify areas of future research.

Jeremy Clark, P. C. van Oorschot, Scott Ruoti et al.

While email is the most ubiquitous and interoperable form of online communication today, it was not conceived with strong security guarantees, and the ensuing security enhancements are, by contrast, lacking in both ubiquity and interoperability. This situation motivates our research. We begin by identifying a variety of stakeholders who have an interest in the current email system and in efforts to provide secure solutions. We then use the tussle among stakeholders to explain the evolution of fragmented secure email solutions undertaken by industry, academia, and independent developers. We also evaluate the building blocks of secure email -- cryptographic primitives, key management schemes, and system designs -- to identify their support for stakeholder properties. From our analysis, we conclude that a one-size-fits-all solution is unlikely. Furthermore, we highlight that vulnerable users are not well served by current solutions, account for the failure of PGP, and argue that secure messaging, while complementary, is not a fully substitutable technology.

Mark O'Neill, Scott Heidbrink, Jordan Whitehead et al.

We describe TrustBase, an architecture that provides certificate-based authentication as an operating system service. TrustBase enforces best practices for certificate validation for all applications and transparently enables existing applications to be strengthened against failures of the CA system. The TrustBase system allows simple deployment of authentication systems that harden the CA system. This enables system administrators, for example, to require certificate revocation checks on all TLS connections, or require STARTTLS for email servers that support it. TrustBase is the first system that is able to secure all TLS traffic, using an approach compatible with all operating systems. We design and evaluate a prototype implementation of TrustBase on Linux, evaluate its security, and demonstrate that it has negligible overhead and universal compatibility with applications. To demonstrate the utility of TrustBase, we have developed six authentication services that strengthen certificate validation for all applications.

Scott Ruoti, Jeff Andersen, Tyler Monson et al.

This paper describes MessageGuard, a browser-based platform for research into usable content-based encryption. MessageGuard is designed to enable collaboration between security and usability researchers on long-standing research questions in this area. It significantly simplifies the effort required to work in this space and provides a place for research results to be shared, replicated, and compared with minimal confounding factors. MessageGuard provides ubiquitous encryption and secure cryptographic operations, enabling research on any existing web application, with realistic usability studies on a secure platform. We validate MessageGuard's compatibility and performance, and we illustrate its utility with case studies for Gmail and Facebook Chat.

Scott Ruoti, Jeff Andersen, Daniel Zappala et al.

This paper presents the results of a laboratory study involving Mailvelope, a modern PGP client that integrates tightly with existing webmail providers. In our study, we brought in pairs of participants and had them attempt to use Mailvelope to communicate with each other. Our results shown that more than a decade and a half after \textit{Why Johnny Can't Encrypt}, modern PGP tools are still unusable for the masses. We finish with a discussion of pain points encountered using Mailvelope, and discuss what might be done to address them in future PGP systems.

Scott Ruoti, Jeff Andersen, Scott Heidbrink et al.

Secure email is increasingly being touted as usable by novice users, with a push for adoption based on recent concerns about government surveillance. To determine whether secure email is for grassroots adoption, we employ a laboratory user study that recruits pairs of novice to install and use several of the latest systems to exchange secure messages. We present quantitative and qualitative results from 25 pairs of novice users as they use Pwm, Tutanota, and Virtru. Participants report being more at ease with this type of study and better able to cope with mistakes since both participants are "on the same page". We find that users prefer integrated solutions over depot-based solutions, and that tutorials are important in helping first-time users. Hiding the details of how a secure email system provides security can lead to a lack of trust in the system. Participants expressed a desire to use secure email, but few wanted to use it regularly and most were unsure of when they might use it.

Scott Ruoti, Jeff Andersen, Travis Hendershot et al.

Private Webmail 2.0 (Pwm 2.0) improves upon the current state of the art by increasing the usability and practical security of secure email for ordinary users. More users are able to send and receive encrypted emails without mistakenly revealing sensitive information. In this paper we describe user interface traits that positively affect the usability and security of Pwm 2.0: (1) an artificial delay to encryption that enhances user confidence in Pwm 2.0 while simultaneously instructing users on who can read their encrypted messages; (2) a modified composition interface that helps protect users from mistakenly sending sensitive information in the clear; (3) an annotated secure email composition interface that instructs users on how to correctly use secure email; and (4) inline, context-sensitive tutorials, which improved view rates for tutorials from less than 10% in earlier systems to over 90% for Pwm 2.0. In a user study involving 51 participants we validate these interface modifications, and also show that the use of manual encryption has no effect on usability or security.

Scott Ruoti, Mark O'Neil, Daniel Zappala et al.

This paper reports the results of a survey of 1,976 individuals regarding their opinions on TLS inspection, a controversial technique that can be used for both benevolent and malicious purposes. Responses indicate that participants hold nuanced opinions on security and privacy trade-offs, with most recognizing legitimate uses for the practice, but also concerned about threats from hackers or government surveillance. There is strong support for notification and consent when a system is intercepting their encrypted traffic, although this support varies depending on the situation. A significant concern about malicious uses of TLS inspection is identity theft, and many would react negatively and some would change their behavior if they discovered inspection occurring without their knowledge. We also find that there are a small but significant number of participants who are jaded by the current state of affairs and have no expectation of privacy.

Mark O'Neill, Scott Ruoti, Kent Seamons et al.

The use of TLS proxies to intercept encrypted traffic is controversial since the same mechanism can be used for both benevolent purposes, such as protecting against malware, and for malicious purposes, such as identity theft or warrantless government surveillance. To understand the prevalence and uses of these proxies, we build a TLS proxy measurement tool and deploy it via Google AdWords campaigns. We generate 15.2 million certificate tests across two large-scale measurement studies. We find that 1 in 250 TLS connections are TLS-proxied. The majority of these proxies appear to be benevolent, however we identify over 3,600 cases where eight malware products are using this technology nefariously. We also find numerous instances of negligent, duplicitous, and suspicious behavior, some of which degrade security for users without their knowledge. Distinguishing these types of practices is challenging in practice, indicating a need for transparency and user awareness.