Battling Misinformation: An Empirical Study on Adversarial Factuality in Open-Source Large Language Models
This addresses the problem of misinformation vulnerability in open-source LLMs for users and developers, but it is incremental as it builds on existing adversarial evaluation methods.
This study tackled the problem of adversarial factuality, where misinformation is deliberately inserted into prompts with varying confidence levels, by empirically evaluating eight open-source large language models (LLMs) on their detection capabilities. The results showed that LLaMA 3.1 (8B) performed robustly, while Falcon (7B) had lower performance, and detection success generally improved with lower adversarial confidence, except for LLaMA 3.1 (8B) and Phi 3 (3.8B) where it decreased.
Adversarial factuality refers to the deliberate insertion of misinformation into input prompts by an adversary, characterized by varying levels of expressed confidence. In this study, we systematically evaluate the performance of several open-source large language models (LLMs) when exposed to such adversarial inputs. Three tiers of adversarial confidence are considered: strongly confident, moderately confident, and limited confidence. Our analysis encompasses eight LLMs: LLaMA 3.1 (8B), Phi 3 (3.8B), Qwen 2.5 (7B), Deepseek-v2 (16B), Gemma2 (9B), Falcon (7B), Mistrallite (7B), and LLaVA (7B). Empirical results indicate that LLaMA 3.1 (8B) exhibits a robust capability in detecting adversarial inputs, whereas Falcon (7B) shows comparatively lower performance. Notably, for the majority of the models, detection success improves as the adversary's confidence decreases; however, this trend is reversed for LLaMA 3.1 (8B) and Phi 3 (3.8B), where a reduction in adversarial confidence corresponds with diminished detection performance. Further analysis of the queries that elicited the highest and lowest rates of successful attacks reveals that adversarial attacks are more effective when targeting less commonly referenced or obscure information.