Roberto Ulloa

Celina Kacperski, Roberto Ulloa, Peter Selb et al.

As citizens increasingly encounter political information in digital environments, understanding whether this engagement shapes their policy views has become a central concern. Drawing on dual-process theories of persuasion, we argue that motivational activation is an enabling condition for policy support change in high-choice online environments. We test this in a three-wave field experiment with German participants (n = 791) across three policy topics (basic child support, renewable energy transition, cannabis legalization), in which participants were randomly assigned to a control group, and two encouragement conditions: a verbal encouragement, or a monetary incentive tied to a knowledge test. Browsing behavior was passively tracked via digital trace data over a 20-hour window. We find that self-directed online information search produced changes in policy support for child support and cannabis legalization but not for the energy transition, with monetary incentives producing significant effects rather than verbal prompts. We discuss motivational salience, issue malleability, and search-environment quality as joint conditions under which political information engagement can produce detectable changes in policy support.

Roberto Ulloa, Tiedemann Leonard, Peter Selb et al.

Persistent inequalities in political knowledge are a central concern in political communication. We organize the mechanisms underlying the knowledge-gap literature by distinguishing between individual preconditions, structural features of the information environment, and topic characteristics. Within this framework, we note that self-directed information seeking, a prototypical form of intentional exposure, has received little attention despite its importance in navigating today's complex information environment. We conducted a field experiment in Germany combining randomized encouragements and passive browser tracking to examine how individuals with varying education levels acquire policy-specific knowledge through online search. Participants were randomly assigned to one of three conditions (verbal encouragement, financial encouragement, or control) to seek information on three salient policy topics differing in divisiveness and complexity (child support, energy transition, and cannabis legalization). We estimate both intention-to-treat (ITT) and local average treatment effects (LATE) of information seeking on post-search knowledge outcomes, with a focus on education and civic knowledge as moderators. While the interventions equalized information-seeking behavior, the results provide some support for the knowledge gap hypothesis: knowledge gains were concentrated among participants with higher education or baseline civic knowledge, who, according to our post-hoc exploratory analyses, appeared more effective at navigating search results. These findings indicate that a narrowing of knowledge inequalities goes beyond motivation: it calls for both individual-level interventions to strengthen citizens' skills and structural-level adaptations to foster more equitable learning environments.

Julian Schelb, Roberto Ulloa, Andreas Spitz

Researchers in the political and social sciences often rely on classification models to analyze trends in information consumption by examining browsing histories of millions of webpages. Automated scalable methods are necessary due to the impracticality of manual labeling. In this paper, we model the detection of topic-related content as a binary classification task and compare the accuracy of fine-tuned pre-trained encoder models against in-context learning strategies. Using only a few hundred annotated data points per topic, we detect content related to three German policies in a database of scraped webpages. We compare multilingual and monolingual models, as well as zero and few-shot approaches, and investigate the impact of negative sampling strategies and the combination of URL & content-based features. Our results show that a small sample of annotated data is sufficient to train an effective classifier. Fine-tuning encoder-based models yields better results than in-context learning. Classifiers using both URL & content-based features perform best, while using URLs alone provides adequate results when content is unavailable.

Roberto Ulloa, Eve M. Zucker, Daniel Bultmann et al.

The proliferation of large language models (LLMs) can influence how historical narratives are disseminated and perceived. This study explores the implications of LLMs' responses on the representation of mass atrocity memory, examining whether generative AI systems contribute to prosthetic memory, i.e., mediated experiences of historical events, or to what we term "prosthetic denial," the AI-mediated erasure or distortion of atrocity memories. We argue that LLMs function as interfaces that can elicit prosthetic memories and, therefore, act as experiential sites for memory transmission, but also introduce risks of denialism, particularly when their outputs align with contested or revisionist narratives. To empirically assess these risks, we conducted a comparative audit of five LLMs (Claude, GPT, Llama, Mixtral, and Gemini) across four historical case studies: the Holodomor, the Holocaust, the Cambodian Genocide, and the genocide against the Tutsis in Rwanda. Each model was prompted with questions addressing common denialist claims in English and an alternative language relevant to each case (Ukrainian, German, Khmer, and French). Our findings reveal that while LLMs generally produce accurate responses for widely documented events like the Holocaust, significant inconsistencies and susceptibility to denialist framings are observed for more underrepresented cases like the Cambodian Genocide. The disparities highlight the influence of training data availability and the probabilistic nature of LLM responses on memory integrity. We conclude that while LLMs extend the concept of prosthetic memory, their unmoderated use risks reinforcing historical denialism, raising ethical concerns for (digital) memory preservation, and potentially challenging the advantageous role of technology associated with the original values of prosthetic memory.

Aleksandra Urman, Mykola Makhortykh, Roberto Ulloa et al.

Web search engines are important online information intermediaries that are frequently used and highly trusted by the public despite multiple evidence of their outputs being subjected to inaccuracies and biases. One form of such inaccuracy, which so far received little scholarly attention, is the presence of conspiratorial information, namely pages promoting conspiracy theories. We address this gap by conducting a comparative algorithm audit to examine the distribution of conspiratorial information in search results across five search engines: Google, Bing, DuckDuckGo, Yahoo and Yandex. Using a virtual agent-based infrastructure, we systematically collect search outputs for six conspiracy theory-related queries (flat earth, new world order, qanon, 9/11, illuminati, george soros) across three locations (two in the US and one in the UK) and two observation periods (March and May 2021). We find that all search engines except Google consistently displayed conspiracy-promoting results and returned links to conspiracy-dedicated websites in their top results, although the share of such content varied across queries. Most conspiracy-promoting results came from social media and conspiracy-dedicated websites while conspiracy-debunking information was shared by scientific websites and, to a lesser extent, legacy media. The fact that these observations are consistent across different locations and time periods highlight the possibility of some search engines systematically prioritizing conspiracy-promoting content and, thus, amplifying their distribution in the online environments.

Mykola Makhortykh, Aleksandra Urman, Roberto Ulloa

Web search engines influence perception of social reality by filtering and ranking information. However, their outputs are often subjected to bias that can lead to skewed representation of subjects such as professional occupations or gender. In our paper, we use a mixed-method approach to investigate presence of race and gender bias in representation of artificial intelligence (AI) in image search results coming from six different search engines. Our findings show that search engines prioritize anthropomorphic images of AI that portray it as white, whereas non-white images of AI are present only in non-Western search engines. By contrast, gender representation of AI is more diverse and less skewed towards a specific gender that can be attributed to higher awareness about gender bias in search outputs. Our observations indicate both the the need and the possibility for addressing bias in representation of societally relevant subjects, such as technological innovation, and emphasize the importance of designing new approaches for detecting bias in information retrieval systems.

Aleksandra Urman, Mykola Makhortykh, Roberto Ulloa

We audit the presence of domain-level source diversity bias in video search results. Using a virtual agent-based approach, we compare outputs of four Western and one non-Western search engines for English and Russian queries. Our findings highlight that source diversity varies substantially depending on the language with English queries returning more diverse outputs. We also find disproportionately high presence of a single platform, YouTube, in top search outputs for all Western search engines except Google. At the same time, we observe that Youtube's major competitors such as Vimeo or Dailymotion do not appear in the sampled Google's video search results. This finding suggests that Google might be downgrading the results from the main competitors of Google-owned Youtube and highlights the necessity for further studies focusing on the presence of own-content bias in Google's search results.