Benchmarking Knowledge Boundary for Large Language Models: A Different Perspective on Model Evaluation
This addresses the need for more dependable and robust evaluation benchmarks for large language models, particularly for researchers and developers assessing knowledge ability across domains, though it is incremental as it builds on existing evaluation frameworks.
The paper tackles the problem of unreliable evaluation of large language models due to prompt sensitivity by introducing the concept of knowledge boundary, which includes both prompt-agnostic and prompt-sensitive knowledge, and proposes a projected gradient descent method with semantic constraints to compute it, demonstrating superior performance compared to existing methods.
In recent years, substantial advancements have been made in the development of large language models, achieving remarkable performance across diverse tasks. To evaluate the knowledge ability of language models, previous studies have proposed lots of benchmarks based on question-answering pairs. We argue that it is not reliable and comprehensive to evaluate language models with a fixed question or limited paraphrases as the query, since language models are sensitive to prompt. Therefore, we introduce a novel concept named knowledge boundary to encompass both prompt-agnostic and prompt-sensitive knowledge within language models. Knowledge boundary avoids prompt sensitivity in language model evaluations, rendering them more dependable and robust. To explore the knowledge boundary for a given model, we propose projected gradient descent method with semantic constraints, a new algorithm designed to identify the optimal prompt for each piece of knowledge. Experiments demonstrate a superior performance of our algorithm in computing the knowledge boundary compared to existing methods. Furthermore, we evaluate the ability of multiple language models in several domains with knowledge boundary.