How to Steal Reasoning Without Reasoning Traces
This work addresses the problem of extracting and transferring reasoning capabilities from black-box LLMs to smaller, open-source models, which is significant for democratizing advanced AI reasoning and reducing reliance on proprietary models.
This paper introduces trace inversion models that can reconstruct detailed reasoning traces from black-box LLMs, given only their inputs, final answers, and optional reasoning summaries. Fine-tuning student models on these inverted traces significantly improves their reasoning capabilities, for instance, boosting Qwen-2.5-7B-Instruct's performance from 56.8% to 77.6% on MATH500 and from 11.7% to 42.3% on JEEBench.
Many large language models (LLMs) use reasoning to generate responses but do not reveal their full reasoning traces (a.k.a. chains of thought), instead outputting only final answers and brief reasoning summaries. To demonstrate that hiding reasoning traces does not prevent users from "stealing" a model's reasoning capabilities, we introduce trace inversion models that, given only the inputs, answers, and (optionally) reasoning summaries exposed by a target model, generate detailed, synthetic reasoning traces. We show that (1) traces synthesized by trace inversion have high overlap with the ground-truth reasoning traces (when available), and (2) fine-tuning student models on inverted traces substantially improves their reasoning. For example, fine-tuning Qwen-2.5-7B-Instruct on traces inverted from the answers and summaries of GPT-5 mini, a commercial black-box LLM, improves its performance from 56.8% to 77.6% on MATH500 and from 11.7% to 42.3% on JEEBench, compared to fine-tuning on just the answers and summaries.