T1: Tool-integrated Self-verification for Test-time Compute Scaling in Small Language Models
This work addresses the challenge of enhancing self-verification in small language models for tasks requiring memorization, offering a tool-based solution that improves performance on mathematical and knowledge-intensive benchmarks, though it is incremental as it builds on existing test-time compute scaling methods.
The paper tackles the problem of small language models (sLMs) struggling with self-verification under test-time compute scaling, particularly for memorization-heavy tasks like numerical calculations and fact-checking, and proposes Tool-integrated self-verification (T1) to delegate such steps to external tools, resulting in a Llama-3.2 1B model outperforming a larger Llama-3.1 8B model on the MATH benchmark.
Recent studies have demonstrated that test-time compute scaling effectively improves the performance of small language models (sLMs). However, prior research has mainly examined test-time compute scaling with an additional larger model as a verifier, leaving self-verification by sLMs underexplored. In this work, we investigate whether sLMs can reliably self-verify their outputs under test-time scaling. We find that even with knowledge distillation from larger verifiers, sLMs struggle with verification tasks requiring memorization, such as numerical calculations and fact-checking. To address this limitation, we propose Tool-integrated self-verification (T1), which delegates memorization-heavy verification steps to external tools, such as a code interpreter. Our theoretical analysis shows that tool integration reduces memorization demands and improves test-time scaling performance. Experiments on the MATH benchmark demonstrate that, with T1, a Llama-3.2 1B model under test-time scaling outperforms the significantly larger Llama-3.1 8B model. Moreover, T1 generalizes effectively to both mathematical (MATH500) and multi-domain knowledge-intensive tasks (MMLU-Pro). Our findings highlight the potential of tool integration to substantially improve the self-verification abilities of sLMs.