Elissa M. Redmiles

Ana-Maria Cretu, Klim Kireev, Amro Abdalla et al.

We evaluate the effectiveness of filtering child images from training datasets of text-to-image models to prevent model misuse to create child sexual abuse material (CSAM). First, we capture the complexity of preventing CSAM generation using a game-based security definition. Second, we show that current detection methods cannot remove all children from a dataset. Third, using an ethical proxy for CSAM (a child wearing glasses), we show that even when only a small percentage of child images are left in the training dataset after filtering, there exist prompting strategies that generate a child wearing glasses using only a few more queries than when the model is trained on the unfiltered data. Fine-tuning the filtered model on child images further reduces the additional query overhead. We also show that re-introducing a concept is possible via fine-tuning even if filtering is perfect. Our results show that current child filtering methods offer limited protection to closed-weight models and no protection to open-weight models, while reducing the generality of the model by hindering the generation of child-related concepts or changing their representation. We conclude by outlining challenges in conducting evaluations that establish robust evidence on the impact of concept filtering defenses for CSAM.

Cassidy Gibson, Daniel Olszewski, Natalie Grace Brigham et al.

Given a source image of a clothed person (an image subject), AI-based nudification applications can produce nude (undressed) images of that person. Moreover, not only do such applications exist, but there is ample evidence of the use of such applications in the real world and without the consent of an image subject. Still, despite the growing awareness of the existence of such applications and their potential to violate the rights of image subjects and cause downstream harms, there has been no systematic study of the nudification application ecosystem across multiple applications. We conduct such a study here, focusing on 20 popular and easy-to-find nudification websites. We study the positioning of these web applications (e.g., finding that most sites explicitly target the nudification of women, not all people), the features that they advertise (e.g., ranging from undressing-in-place to the rendering of image subjects in sexual positions, as well as differing user-privacy options), and their underlying monetization infrastructure (e.g., credit cards and cryptocurrencies). We believe this work will empower future, data-informed conversations -- within the scientific, technical, and policy communities -- on how to better protect individuals' rights and minimize harm in the face of modern (and future) AI-based nudification applications. Content warning: This paper includes descriptions of web applications that can be used to create synthetic non-consensual explicit AI-created imagery (SNEACI). This paper also includes an artistic rendering of a user interface for such an application.

Jaron Mink, Lucy Qin, Elissa M. Redmiles

AI-generated media is radically changing the way content is both consumed and produced on the internet, and in no place is this potentially more visible than in sexual content. AI-generated sexual content (AIG-SC) is increasingly enabled by an ecosystem of individual AI developers, specialized third-party applications, and foundation model providers. AIG-SC raises a number of concerns from old debates about the line between pornography and obscenity, to newer debates about fair use and labor displacement (in this case, of sex workers), and spurred new regulations to curb the spread of non-consensual intimate imagery (NCII) created using the same technology used to create AIG-SC. However, despite the growing prevalence of AIG-SC, little is known about its creators, their motivations, and what types of content they produce. To inform effective governance in this space, we perform an in-depth study to understand what AIG-SC creators make, along with how and why they make it. Interviews of 28 AIG-SC creators, ranging from hobbyists to entrepreneurs to those who moderate communities of hundreds of thousands of other creators, reveal a wide spectrum of motivations, including sexual exploration, creative expression, technical experimentation, and in a handful of cases, the creation of NCII.

Angelica Goetzen, Samuel Dooley, Elissa M. Redmiles

People's privacy sentiments influence changes in legislation as well as technology design and use. While single-point-in-time investigations of privacy sentiment offer useful insight, study of people's privacy sentiments over time is also necessary to better understand and anticipate evolving privacy attitudes. In this work, we use repeated cross-sectional surveys (n=6,676) to model the sentiments of people in the U.S. toward collection and use of data for government- and health-related purposes from 2019-2021. After the onset of COVID-19, we observe significant decreases in respondent acceptance of government data use and significant increases in acceptance of health-related data uses. While differences in privacy attitudes between sociodemographic groups largely decreased over this time period, following the 2020 U.S. national elections, we observe some of the first evidence that privacy sentiments may change based on the alignment between a user's politics and the political party in power. Our results offer insight into how privacy attitudes may have been impacted by recent events and allow us to identify potential predictors of changes in privacy attitudes during times of geopolitical or national change.

Rachel Cummings, Gabriel Kaptchuk, Elissa M. Redmiles

Despite recent widespread deployment of differential privacy, relatively little is known about what users think of differential privacy. In this work, we seek to explore users' privacy expectations related to differential privacy. Specifically, we investigate (1) whether users care about the protections afforded by differential privacy, and (2) whether they are therefore more willing to share their data with differentially private systems. Further, we attempt to understand (3) users' privacy expectations of the differentially private systems they may encounter in practice and (4) their willingness to share data in such systems. To answer these questions, we use a series of rigorously conducted surveys (n=2424). We find that users care about the kinds of information leaks against which differential privacy protects and are more willing to share their private information when the risks of these leaks are less likely to happen. Additionally, we find that the ways in which differential privacy is described in-the-wild haphazardly set users' privacy expectations, which can be misleading depending on the deployment. We synthesize our results into a framework for understanding a user's willingness to share information with differentially private systems, which takes into account the interaction between the user's prior privacy concerns and how differential privacy is described.

Sean Kross, Eszter Hargittai, Elissa M. Redmiles

Hundreds of millions of people learn something new online every day. Simultaneously, the study of online education has blossomed within the human computer interaction community, with new systems, experiments, and observations creating and exploring previously undiscovered online learning environments. In this study we endeavor to characterize this entire landscape of online learning experiences using a national survey of 2260 US adults who are balanced to match the demographics of the U.S. We examine the online learning resources that they consult, and we analyze the subjects that they pursue using those resources. Furthermore, we compare both formal and informal online learning experiences on a larger scale than has ever been done before, to our knowledge, to better understand which subjects people are seeking for intensive study. We find that there is a core set of online learning experiences that are central to other experiences and these are shared among the majority of people who learn online. We conclude by showing how looking outside of these core online learning experiences can reveal opportunities for innovation in online education.

Imani N. Sherman, Elissa M. Redmiles, Jack W. Stokes

The growth of misinformation technology necessitates the need to identify fake videos. One approach to preventing the consumption of these fake videos is provenance which allows the user to authenticate media content to its original source. This research designs and investigates the use of provenance indicators to help users identify fake videos. We first interview users regarding their experiences with different misinformation modes (text, image, video) to guide the design of indicators within users' existing perspectives. Then, we conduct a participatory design study to develop and design fake video indicators. Finally, we evaluate participant-designed indicators via both expert evaluations and quantitative surveys with a large group of end-users. Our results provide concrete design guidelines for the emerging issue of fake videos. Our findings also raise concerns regarding users' tendency to overgeneralize from misinformation warning messages, suggesting the need for further research on warning design in the ongoing fight against misinformation.

Gabriel Kaptchuk, Daniel G. Goldstein, Eszter Hargittai et al.

A growing number of contact tracing apps are being developed to complement manual contact tracing. A key question is whether users will be willing to adopt these contact tracing apps. In this work, we survey over 4,500 Americans to evaluate (1) the effect of both accuracy and privacy concerns on reported willingness to install COVID19 contact tracing apps and (2) how different groups of users weight accuracy vs. privacy. Drawing on our findings from these first two research questions, we (3) quantitatively model how the amount of public health benefit (reduction in infection rate), amount of individual benefit (true-positive detection of exposures to COVID), and degree of privacy risk in a hypothetical contact tracing app may influence American's willingness to install. Our work takes a descriptive ethics approach toward offering implications for the development of policy and app designs related to COVID19.

Nina Grgić-Hlača, Gabriel Lima, Adrian Weller et al.

A growing number of oversight boards and regulatory bodies seek to monitor and govern algorithms that make decisions about people's lives. Prior work has explored how people believe algorithmic decisions should be made, but there is little understanding of how individual factors like sociodemographics or direct experience with a decision-making scenario may affect their ethical views. We take a step toward filling this gap by exploring how people's perceptions of one aspect of procedural algorithmic fairness (the fairness of using particular features in an algorithmic decision) relate to their (i) demographics (age, education, gender, race, political views) and (ii) personal experiences with the algorithmic decision-making scenario. We find that political views and personal experience with the algorithmic decision context significantly influence perceptions about the fairness of using different features for bail decision-making. Drawing on our results, we discuss the implications for stakeholder engagement and algorithmic oversight including the need to consider multiple dimensions of diversity in composing oversight and regulatory bodies.

Elissa M. Redmiles

The COVID19 pandemic spread across the world in late 2019 and early 2020. As the pandemic spread, technologists joined forces with public health officials to develop apps to support COVID19 response. Yet, for these technological solutions to benefit public health, users must be willing to adopt these apps.This paper details the potential inputs to a user's decision to adopt a COVID19 contact-tracing app or other technology and empirically validates the relevance of these inputs via both the literature and a demographically-representative survey of 1,000 Americans.

Elissa M. Redmiles

Digital security technology is able to identify and prevent many threats to users accounts. However, some threats remain that, to provide reliable security, require human intervention: e.g., through users paying attention to warning messages or completing secondary authentication procedures. While prior work has broadly explored people's mental models of digital security threats, we know little about users' precise, in-the-moment response process to in-the-wild threats. In this work, we conduct a series of qualitative interviews (n=67) with users who had recently experienced suspicious login incidents on their real Facebook accounts in order to explore this process of account security incident response. We find a common process across participants from five countries -- with differing online and offline cultures -- allowing us to identify areas for future technical development to best support user security. We provide additional insights on the unique nature of incident-response information seeking, known attacker threat models, and lessons learned from a large, cross-cultural qualitative study of digital security.

Elissa M. Redmiles, Michelle L. Mazurek, John P. Dickerson

Accurately modeling human decision-making in security is critical to thinking about when, why, and how to recommend that users adopt certain secure behaviors. In this work, we conduct behavioral economics experiments to model the rationality of end-user security decision-making in a realistic online experimental system simulating a bank account. We ask participants to make a financially impactful security choice, in the face of transparent risks of account compromise and benefits offered by an optional security behavior (two-factor authentication). We measure the cost and utility of adopting the security behavior via measurements of time spent executing the behavior and estimates of the participant's wage. We find that more than 50% of our participants made rational (e.g., utility optimal) decisions, and we find that participants are more likely to behave rationally in the face of higher risk. Additionally, we find that users' decisions can be modeled well as a function of past behavior (anchoring effects), knowledge of costs, and to a lesser extent, users' awareness of risks and context (R2=0.61). We also find evidence of endowment effects, as seen in other areas of economic and psychological decision-science literature, in our digital-security setting. Finally, using our data, we show theoretically that a "one-size-fits"-all emphasis on security can lead to market losses, but that adoption by a subset of users with higher risks or lower costs can lead to market gains.

Nina Grgić-Hlača, Elissa M. Redmiles, Krishna P. Gummadi et al.

As algorithms are increasingly used to make important decisions that affect human lives, ranging from social benefit assignment to predicting risk of criminal recidivism, concerns have been raised about the fairness of algorithmic decision making. Most prior works on algorithmic fairness normatively prescribe how fair decisions ought to be made. In contrast, here, we descriptively survey users for how they perceive and reason about fairness in algorithmic decision making. A key contribution of this work is the framework we propose to understand why people perceive certain features as fair or unfair to be used in algorithms. Our framework identifies eight properties of features, such as relevance, volitionality and reliability, as latent considerations that inform people's moral judgments about the fairness of feature use in decision-making algorithms. We validate our framework through a series of scenario-based surveys with 576 people. We find that, based on a person's assessment of the eight latent properties of a feature in our exemplar scenario, we can accurately (> 85%) predict if the person will judge the use of the feature as fair. Our findings have important implications. At a high-level, we show that people's unfairness concerns are multi-dimensional and argue that future studies need to address unfairness concerns beyond discrimination. At a low-level, we find considerable disagreements in people's fairness judgments. We identify root causes of the disagreements, and note possible pathways to resolve them.