Investigating LLMs as Voting Assistants via Contextual Augmentation: A Case Study on the European Parliament Elections 2024
This addresses the problem of providing automated voting assistance for voters, though it is incremental as it builds on existing LLM and RAG methods for a specific domain.
The study investigated whether large language models (LLMs) like MISTRAL and MIXTRAL can serve as Voting Advice Applications (VAAs) for the 2024 European Parliament elections, finding that MIXTRAL achieved 82% accuracy on average in predicting party stances, with context augmentation via expert-curated information boosting performance by approximately 9%.
In light of the recent 2024 European Parliament elections, we are investigating if LLMs can be used as Voting Advice Applications (VAAs). We audit MISTRAL and MIXTRAL models and evaluate their accuracy in predicting the stance of political parties based on the latest "EU and I" voting assistance questionnaire. Furthermore, we explore alternatives to improve models' performance by augmenting the input context via Retrieval-Augmented Generation (RAG) relying on web search, and Self-Reflection using staged conversations that aim to re-collect relevant content from the model's internal memory. We find that MIXTRAL is highly accurate with an 82% accuracy on average with a significant performance disparity across different political groups (50-95%). Augmenting the input context with expert-curated information can lead to a significant boost of approx. 9%, which remains an open challenge for automated RAG approaches, even considering curated content.