Rachel Xu

Rachel Xu, Nhu Le, Rebekah Park et al.

We conducted in-person ethnography in India and the US to investigate how young people (18-24) trusted online content, just as generative AI (genAI) became mainstream. We found that when online, how participants determined what content to trust was shaped by emotional states, which we term "information modes." Our participants reflexively shifted between modes to maintain "emotional equilibrium," and eschewed engaging literacy skills in the more passive modes in which they spent the most time. We found participants imported trust heuristics from established online contexts into emerging ones (i.e., genAI). This led them to use ill-fitting trust heuristics, and exposed them to the risk of trusting false and misleading information. While many had reservations about AI, prioritizing efficiency, they used genAI and habitual heuristics to quickly achieve goals at the expense of accuracy. We conclude that literacy interventions designed to match users' distinct information modes will be most effective.

Nahema Marchal, Rachel Xu, Rasmi Elasmar et al.

Generative, multimodal artificial intelligence (GenAI) offers transformative potential across industries, but its misuse poses significant risks. Prior research has shed light on the potential of advanced AI systems to be exploited for malicious purposes. However, we still lack a concrete understanding of how GenAI models are specifically exploited or abused in practice, including the tactics employed to inflict harm. In this paper, we present a taxonomy of GenAI misuse tactics, informed by existing academic literature and a qualitative analysis of approximately 200 observed incidents of misuse reported between January 2023 and March 2024. Through this analysis, we illuminate key and novel patterns in misuse during this time period, including potential motivations, strategies, and how attackers leverage and abuse system capabilities across modalities (e.g. image, text, audio, video) in the wild.