MillStone: How Open-Minded Are LLMs?
This addresses the issue of bias and manipulability in LLM-based information systems for users relying on them for controversial topics, though it is incremental as it builds on existing concerns with benchmarks.
The authors tackled the problem of understanding how external arguments influence the stances of large language models (LLMs) on controversial issues, finding that LLMs are generally open-minded and can be easily swayed by authoritative sources, highlighting risks of manipulation.
Large language models equipped with Web search, information retrieval tools, and other agentic capabilities are beginning to supplant traditional search engines. As users start to rely on LLMs for information on many topics, including controversial and debatable issues, it is important to understand how the stances and opinions expressed in LLM outputs are influenced by the documents they use as their information sources. In this paper, we present MillStone, the first benchmark that aims to systematically measure the effect of external arguments on the stances that LLMs take on controversial issues (not all of them political). We apply MillStone to nine leading LLMs and measure how ``open-minded'' they are to arguments supporting opposite sides of these issues, whether different LLMs agree with each other, which arguments LLMs find most persuasive, and whether these arguments are the same for different LLMs. In general, we find that LLMs are open-minded on most issues. An authoritative source of information can easily sway an LLM's stance, highlighting the importance of source selection and the risk that LLM-based information retrieval and search systems can be manipulated.