View From Above: A Framework for Evaluating Distribution Shifts in Model Behavior
This addresses the issue of ensuring reliable AI behavior for users and developers, though it is incremental as it builds on existing evaluation methods.
The paper tackles the problem of evaluating whether large language models' learned representations align with reality by proposing a domain-agnostic framework to detect distribution shifts in their decision-making processes, and it finds statistically significant evidence of behavioral misalignment in over 1,000 trials in a blackjack environment.
When large language models (LLMs) are asked to perform certain tasks, how can we be sure that their learned representations align with reality? We propose a domain-agnostic framework for systematically evaluating distribution shifts in LLMs decision-making processes, where they are given control of mechanisms governed by pre-defined rules. While individual LLM actions may appear consistent with expected behavior, across a large number of trials, statistically significant distribution shifts can emerge. To test this, we construct a well-defined environment with known outcome logic: blackjack. In more than 1,000 trials, we uncover statistically significant evidence suggesting behavioral misalignment in the learned representations of LLM.