Notes to Self: Can LLMs Benefit from Experiential Abstractions?
For LLM reasoning, this work shows that models can autonomously distill and reuse experiential knowledge, offering a practical method to improve performance without external supervision.
This paper investigates whether LLMs can benefit from experiential abstractions—natural-language strategies extracted from solution traces. Using abstraction-augmented prompts, they achieve up to 5% improvement on MATH and GSM8K benchmarks, with self-extracted abstractions matching teacher-extracted ones.
Humans distill experience into reusable abstractions, e.g., strategies and cautionary reminders, and apply them to gradually solve problems more effectively. We study whether Large Language Models (LLMs) can similarly benefit from such experiential abstractions. From LLMs' solution traces on the MATH training set, a stronger teacher or the LLMs themselves extract natural-language abstractions into a retrievable library. We explore two usage modes: (1) inference-time retrieval and (2) reinforcement learning (RL) with abstraction-augmented training prompts. Experiential abstractions improve LLM performance on mathematical and logical reasoning benchmarks. Self-extracted abstractions match teacher-extracted ones, and our abstraction usage framework can transfer to other datasets and models. These findings suggest LLMs can extract and apply experiential abstractions much as humans leverage distilled experience.