Polaris: A Gödel Agent Framework for Small Language Models through Experience-Abstracted Policy Repair
This work addresses the challenge of achieving persistent and reusable policy-level improvements in compact models, which is incremental as it builds on existing Gödel agent concepts but applies them to small-scale settings with novel abstraction techniques.
The paper tackles the problem of enabling small language models to perform recursive self-improvement by introducing Polaris, a Gödel agent framework that uses experience abstraction for policy repair, resulting in consistent performance gains across multiple benchmarks including arithmetic reasoning and graduate-level problem solving.
Gödel agent realize recursive self-improvement: an agent inspects its own policy and traces and then modifies that policy in a tested loop. We introduce Polaris, a Gödel agent for compact models that performs policy repair via experience abstraction, turning failures into policy updates through a structured cycle of analysis, strategy formation, abstraction, and minimal code pat ch repair with conservative checks. Unlike response level self correction or parameter tuning, Polaris makes policy level changes with small, auditable patches that persist in the policy and are reused on unseen instances within each benchmark. As part of the loop, the agent engages in meta reasoning: it explains its errors, proposes concrete revisions to its own policy, and then updates the policy. To enable cumulative policy refinement, we introduce experience abstraction, which distills failures into compact, reusable strategies that transfer to unseen instances. On MGSM, DROP, GPQA, and LitBench (covering arithmetic reasoning, compositional inference, graduate-level problem solving, and creative writing evaluation), a 7-billion-parameter model equipped with Polaris achieves consistent gains over the base policy and competitive baselines.