Think Again! The Effect of Test-Time Compute on Preferences, Opinions, and Beliefs of Large Language Models
This addresses a critical issue for users and developers of LLMs, as it reveals concerning trends in model behavior that could influence decision-making, though it is incremental in benchmarking and analysis.
The paper tackles the problem of evaluating subjective preferences, opinions, and beliefs in Large Language Models (LLMs) by introducing the POBs benchmark, and finds that increasing test-time compute offers only limited gains while newer models are becoming less consistent and more biased.
As Large Language Models (LLMs) become deeply integrated into human life and increasingly influence decision-making, it's crucial to evaluate whether and to what extent they exhibit subjective preferences, opinions, and beliefs. These tendencies may stem from biases within the models, which may shape their behavior, influence the advice and recommendations they offer to users, and potentially reinforce certain viewpoints. This paper presents the Preference, Opinion, and Belief survey (POBs), a benchmark developed to assess LLMs' subjective inclinations across societal, cultural, ethical, and personal domains. We applied our benchmark to evaluate leading open- and closed-source LLMs, measuring desired properties such as reliability, neutrality, and consistency. In addition, we investigated the effect of increasing the test-time compute, through reasoning and self-reflection mechanisms, on those metrics. While effective in other tasks, our results show that these mechanisms offer only limited gains in our domain. Furthermore, we reveal that newer model versions are becoming less consistent and more biased toward specific viewpoints, highlighting a blind spot and a concerning trend. POBS: https://ibm.github.io/POBS