Is More Privileged Information Better? From Solution Traces to Problem-Solving Structure in Self-Distilled Reasoning
For researchers working on self-distillation for reasoning, this work shows that the representation of privileged information matters, offering a new design choice that improves performance.
The paper proposes Problem-Space-Guided OPSD (PS-OPSD), which replaces complete reference solutions with trajectory-grounded guidance in self-distilled reasoning. Across three math benchmarks and models from 1.7B to 8B, PS-OPSD achieves the highest aggregate question-only accuracy among compared methods.
On-policy self-distillation (OPSD) improves reasoning by using a privileged view of a model conditioned on reference solutions to supervise a student view that observes only the question. However, the teacher-provided token-level targets may depend on reference-specific information unavailable at inference time. We propose Problem-Space-Guided OPSD (PS-OPSD), which replaces the complete solution with trajectory-grounded guidance describing the initial state, goal conditions, constraints, and a selected state-transition path. The student rollout and OPSD objective remain unchanged. Across three mathematical reasoning benchmarks and model scales ranging from 1.7B to 8B, PS-OPSD achieves the highest aggregate question-only accuracy among the compared methods. Controlled experiments further indicate that guidance relevance and path coherence contribute to these gains, highlighting the representation of privileged information as an important design choice in OPSD.