CYJul 26, 2024
Surveys Considered Harmful? Reflecting on the Use of Surveys in AI Research, Development, and GovernanceMohammmad Tahaei, Daricia Wilkinson, Alisa Frik et al.
Calls for engagement with the public in Artificial Intelligence (AI) research, development, and governance are increasing, leading to the use of surveys to capture people's values, perceptions, and experiences related to AI. In this paper, we critically examine the state of human participant surveys associated with these topics. Through both a reflexive analysis of a survey pilot spanning six countries and a systematic literature review of 44 papers featuring public surveys related to AI, we explore prominent perspectives and methodological nuances associated with surveys to date. We find that public surveys on AI topics are vulnerable to specific Western knowledge, values, and assumptions in their design, including in their positioning of ethical concepts and societal values, lack sufficient critical discourse surrounding deployment strategies, and demonstrate inconsistent forms of transparency in their reporting. Based on our findings, we distill provocations and heuristic questions for our community, to recognize the limitations of surveys for meeting the goals of engagement, and to cultivate shared principles to design, deploy, and interpret surveys cautiously and responsibly.
51.2HCMar 10
Privacy and Safety Experiences and Concerns of U.S. Women Using Generative AI for Seeking Sexual and Reproductive Health InformationIna Kaleva, Xiao Zhan, Ruba Abu-Salma et al.
The rapid adoption of generative AI (GenAI) chatbots has reshaped access to sexual and reproductive health (SRH) information, particularly following the overturning of Roe v. Wade, as individuals assigned female at birth increasingly turn to online sources. However, existing research remains largely model-centered, paying limited attention to user privacy and safety. We conducted semi-structured interviews with 18 U.S.-based participants from both restrictive and non-restrictive states who had used GenAI chatbots to seek SRH information. Adoption was influenced by perceived utility, usability, credibility, accessibility, and anthropomorphism, and many participants disclosed sensitive personal SRH details. Participants identified multiple privacy risks, including excessive data collection, government surveillance, profiling, model training, and data commodification. While most participants accepted these risks in exchange for perceived utility, abortion-related queries elicited heightened safety concerns. Few participants employed protective strategies beyond minimizing disclosures or deleting data. Based on these findings, we offer design and policy recommendations, such as health-specific features and stronger moderation practices, to enhance privacy and safety in GenAI-supported SRH information seeking.
HCOct 6, 2021
Cookie Banners, What's the Purpose? Analyzing Cookie Banner Text Through a Legal LensCristiana Santos, Arianna Rossi, Lorena Sánchez Chamorro et al.
A cookie banner pops up when a user visits a website for the first time, requesting consent to the use of cookies and other trackers for a variety of purposes. Unlike prior work that has focused on evaluating the user interface (UI) design of cookie banners, this paper presents an in-depth analysis of what cookie banners say to users to get their consent. We took an interdisciplinary approach to determining what cookie banners should say. Following the legal requirements of the ePrivacy Directive (ePD) and the General Data Protection Regulation (GDPR), we manually annotated around 400 cookie banners presented on the most popular English-speaking websites visited by users residing in the EU. We focused on analyzing the purposes of cookie banners and how these purposes were expressed (e.g., any misleading or vague language, any use of jargon). We found that 89% of cookie banners violated applicable laws. In particular, 61% of banners violated the purpose specificity requirement by mentioning vague purposes, including "user experience enhancement". Further, 30% of banners used positive framing, breaching the freely given and informed consent requirements. Based on these findings, we provide recommendations that regulators can find useful. We also describe future research directions.
HCNov 20, 2018
Evaluating the End-User Experience of Private Browsing ModeRuba Abu-Salma, Benjamin Livshits
Nowadays, all major web browsers have a private browsing mode. However, the mode's benefits and limitations are not particularly understood. Through the use of survey studies, prior work has found that most users are either unaware of private browsing or do not use it. Further, those who do use private browsing generally have misconceptions about what protection it provides. However, prior work has not investigated \emph{why} users misunderstand the benefits and limitations of private browsing. In this work, we do so by designing and conducting a three-part study: (1) an analytical approach combining cognitive walkthrough and heuristic evaluation to inspect the user interface of private mode in different browsers; (2) a qualitative, interview-based study to explore users' mental models of private browsing and its security goals; (3) a participatory design study to investigate why existing browser disclosures, the in-browser explanations of private browsing mode, do not communicate the security goals of private browsing to users. Participants critiqued the browser disclosures of three web browsers: Brave, Firefox, and Google Chrome, and then designed new ones. We find that the user interface of private mode in different web browsers violates several well-established design guidelines and heuristics. Further, most participants had incorrect mental models of private browsing, influencing their understanding and usage of private mode. Additionally, we find that existing browser disclosures are not only vague, but also misleading. None of the three studied browser disclosures communicates or explains the primary security goal of private browsing. Drawing from the results of our user study, we extract a set of design recommendations that we encourage browser designers to validate, in order to design more effective and informative browser disclosures related to private mode.