The Self-Execution Benchmark: Measuring LLMs' Attempts to Overcome Their Lack of Self-Execution
This addresses a fundamental limitation in LLMs' self-awareness for AI researchers, though it is incremental as it builds on existing evaluation methods.
The paper tackles the problem of evaluating whether large language models (LLMs) can predict aspects of their own responses, such as difficulty or refusal, by introducing the Self-Execution Benchmark, and finds that models perform poorly with no consistent improvement from increased size or capability.
Large language models (LLMs) are commonly evaluated on tasks that test their knowledge or reasoning abilities. In this paper, we explore a different type of evaluation: whether an LLM can predict aspects of its own responses. Since LLMs lack the ability to execute themselves, we introduce the Self-Execution Benchmark, which measures a model's ability to anticipate properties of its output, such as whether a question will be difficult for it, whether it will refuse to answer, or what kinds of associations it is likely to produce. Our experiments show that models generally perform poorly on this benchmark, and that increased model size or capability does not consistently lead to better performance. These results suggest a fundamental limitation in how LLMs represent and reason about their own behavior.