Intuitive or Dependent? Investigating LLMs' Behavior Style to Conflicting Prompts
This work addresses understanding LLM decision mechanisms for applications like retrieval-augmented generation, but it is incremental as it builds on existing cognitive theory and benchmarking approaches.
The study investigated how Large Language Models (LLMs) behave when given conflicting prompts versus their internal memory, categorizing their preferences into dependent, intuitive, and rational/irrational styles, and found that role play interventions can change these styles with varying adaptivity across models.
This study investigates the behaviors of Large Language Models (LLMs) when faced with conflicting prompts versus their internal memory. This will not only help to understand LLMs' decision mechanism but also benefit real-world applications, such as retrieval-augmented generation (RAG). Drawing on cognitive theory, we target the first scenario of decision-making styles where there is no superiority in the conflict and categorize LLMs' preference into dependent, intuitive, and rational/irrational styles. Another scenario of factual robustness considers the correctness of prompt and memory in knowledge-intensive tasks, which can also distinguish if LLMs behave rationally or irrationally in the first scenario. To quantify them, we establish a complete benchmarking framework including a dataset, a robustness evaluation pipeline, and corresponding metrics. Extensive experiments with seven LLMs reveal their varying behaviors. And, with role play intervention, we can change the styles, but different models present distinct adaptivity and upper-bound. One of our key takeaways is to optimize models or the prompts according to the identified style. For instance, RAG models with high role play adaptability may dynamically adjust the interventions according to the quality of retrieval results -- being dependent to better leverage informative context; and, being intuitive when external prompt is noisy.