Finding Answers in Thought Matters: Revisiting Evaluation on Large Language Models with Reasoning
This addresses the need for more reliable evaluation methods in AI research, particularly for reasoning models, though it is incremental as it builds on existing extraction techniques.
The paper tackles the problem of evaluating large language models (LLMs) in reasoning tasks by showing that answer extraction algorithms significantly affect performance and answer distributions, and proposes an Answer Regeneration framework that improves robustness and performance across math and open-ended QA tasks.
Evaluating generative models, such as large language models (LLMs), commonly involves question-answering tasks where the final answer is selected based on probability of answer choices. On the other hand, for models requiring reasoning, the method of answer extraction plays a critical role. Our research reveals that the performance of reasoning models and their final answer distributions are highly sensitive to the answer extraction algorithm employed. In order to mitigate this, we propose a basic framework: Answer Regeneration. The method uses an additional model inference, providing the prior input and output prefaced by the prompt "Answer:". The final answer is then selected or extracted from the regenerated output. We show that this extraction-rule-agnostic approach exhibits improved performance and enhanced robustness. Furthermore, we have applied this framework to general math problems and open-ended question answering tasks. Our analysis and this framework could offer a more reliable results for model evaluation.