Marcelo Sartori Locatelli

2papers

2 Papers

CLAug 9, 2024
Examining the Behavior of LLM Architectures Within the Framework of Standardized National Exams in Brazil

Marcelo Sartori Locatelli, Matheus Prado Miranda, Igor Joaquim da Silva Costa et al.

The Exame Nacional do Ensino Médio (ENEM) is a pivotal test for Brazilian students, required for admission to a significant number of universities in Brazil. The test consists of four objective high-school level tests on Math, Humanities, Natural Sciences and Languages, and one writing essay. Students' answers to the test and to the accompanying socioeconomic status questionnaire are made public every year (albeit anonymized) due to transparency policies from the Brazilian Government. In the context of large language models (LLMs), these data lend themselves nicely to comparing different groups of humans with AI, as we can have access to human and machine answer distributions. We leverage these characteristics of the ENEM dataset and compare GPT-3.5 and 4, and MariTalk, a model trained using Portuguese data, to humans, aiming to ascertain how their answers relate to real societal groups and what that may reveal about the model biases. We divide the human groups by using socioeconomic status (SES), and compare their answer distribution with LLMs for each question and for the essay. We find no significant biases when comparing LLM performance to humans on the multiple-choice Brazilian Portuguese tests, as the distance between model and human answers is mostly determined by the human accuracy. A similar conclusion is found by looking at the generated text as, when analyzing the essays, we observe that human and LLM essays differ in a few key factors, one being the choice of words where model essays were easily separable from human ones. The texts also differ syntactically, with LLM generated essays exhibiting, on average, smaller sentences and less thought units, among other differences. These results suggest that, for Brazilian Portuguese in the ENEM context, LLM outputs represent no group of humans, being significantly different from the answers from Brazilian students across all tests.

43.5SIApr 27
Mapping Emerging Climate Misinformation Playbooks in the Global South

Marcelo Sartori Locatelli, Wenchao Dong, Pedro Loures Alzamora et al.

Climate misinformation continues to erode support for climate action, a challenge that is especially acute in the Global South, where high climate vulnerability intersects with development pressures. In rapidly evolving digital ecosystems, misinformation adapts to platform incentives, shifting from overt rejection of climate science toward more subtle narratives that contest proposed solutions. This study integrates large-scale platform data with qualitative content analysis to examine how information systems shape contemporary climate discourse. Using a dataset of 226,775 climate-related YouTube videos from Brazil (2019-2025), we identify two dominant misinformation strategies: traditional denial that disputes scientific evidence and an emerging "new denial" that accepts climate change while undermining mitigation and adaptation policies. We find a pronounced transition to solution-focused narratives that target renewable energy, climate governance, and environmental advocates. New denial content is produced by a wider array of actors, attracts higher engagement, and employs more sophisticated persuasive techniques. These patterns disproportionately affect regions already facing structural inequities and bring broader concerns about platform accountability in unequal information environments and suggest the need for governance approaches capable of addressing new denial, a rapidly adapting form of harmful content that often evades existing moderation policies.