BASIL: Bayesian Assessment of Sycophancy in LLMs
This addresses the issue of sycophancy in LLMs for human-AI collaboration in subjective domains like health and law, representing an incremental advance by extending normative assessment beyond objective tasks.
The paper tackled the problem of sycophancy in LLMs by introducing a Bayesian framework to assess its normative effects on rationality without ground-truth data, finding significant evidence of sycophancy in 7 of 8 baselines and that it reduces rationality in 2 out of 4 baselines when probed for probabilities.
Sycophancy (overly agreeable or flattering behavior) is critical to understand in the context of human-AI collaboration, especially in decision-making settings like health, law, and education. Existing methods for studying sycophancy in LLMs are either descriptive (study behavior change when sycophancy is elicited) or normative (provide values-based judgment on behavior change). Together, these approaches help us understand the extent, and impacts, of sycophancy. However, existing normative approaches only apply for objective tasks where ground-truth data exists, ignoring the natural subjectivity in many NLP tasks. Drawing from behavioral economics and rational decision theory, we introduce an Bayesian framework to study the normative effects of sycophancy on rationality in LLMs, without requiring labeled ground-truth. Using this interdisciplinary framework, we study sycophantic behavior in multiple LLM baselines across three different tasks, experimenting with various methods for eliciting sycophancy and obtaining probability judgments from LLMs. We find significant evidence of sycophancy in our experiments (7 of 8 baselines for one of our probing techniques), and observe that sycophancy is more likely to reduce rationality than it is to increase rationality in LLMs' decisions when they are directly probed for probabilities (2 out of 4 baselines show significant increases overall).