On The Truthfulness of 'Surprisingly Likely' Responses of Large Language Models
This work addresses the issue of truthfulness in AI-generated text for users relying on language models for accurate information, representing an incremental improvement over existing methods.
The paper tackles the problem of improving the truthfulness of large language model responses by defining 'surprisingly likely' textual responses, inspired by crowd wisdom principles, and shows that these responses are more accurate on benchmarks like TruthfulQA, with up to 24 percentage points aggregate improvement and up to 70 percentage points on individual categories.
The principle of rewarding a crowd for surprisingly common answers has been used in the literature for designing a number of truthful information elicitation mechanisms. A related method has also been proposed in the literature for better aggregation of crowd wisdom. Drawing a comparison between crowd based collective intelligence systems and large language models, we define the notion of 'surprisingly likely' textual response of a large language model. This notion is inspired by the surprisingly common principle, but tailored for text in a language model. Using benchmarks such as TruthfulQA and openly available LLMs: GPT-2 and LLaMA-2, we show that the surprisingly likely textual responses of large language models are more accurate in many cases compared to standard baselines. For example, we observe up to 24 percentage points aggregate improvement on TruthfulQA and up to 70 percentage points improvement on individual categories of questions in this benchmark. We also provide further analysis of the results, including the cases when surprisingly likely responses are less or not more accurate.