Investigating the Effects of Cognitive Biases in Prompts on Large Language Model Outputs
This addresses a problem for AI developers and users by highlighting the need for bias-aware prompt design to enhance robustness in diverse domains, though it is incremental as it builds on existing bias research.
The paper investigates how cognitive biases in prompts affect Large Language Model outputs, finding that even subtle biases significantly alter answer choices across benchmark datasets, with attention weight analysis showing altered internal decision-making processes.
This paper investigates the influence of cognitive biases on Large Language Models (LLMs) outputs. Cognitive biases, such as confirmation and availability biases, can distort user inputs through prompts, potentially leading to unfaithful and misleading outputs from LLMs. Using a systematic framework, our study introduces various cognitive biases into prompts and assesses their impact on LLM accuracy across multiple benchmark datasets, including general and financial Q&A scenarios. The results demonstrate that even subtle biases can significantly alter LLM answer choices, highlighting a critical need for bias-aware prompt design and mitigation strategy. Additionally, our attention weight analysis highlights how these biases can alter the internal decision-making processes of LLMs, affecting the attention distribution in ways that are associated with output inaccuracies. This research has implications for Al developers and users in enhancing the robustness and reliability of Al applications in diverse domains.