Distilling Feedback into Memory-as-a-Tool
This addresses the problem of computational efficiency for users of large language models in rubric-based learning tasks, though it appears incremental as it builds on existing refinement and memory techniques.
The paper tackles the problem of high inference costs in test-time refinement pipelines by proposing a framework that converts transient critiques into retrievable guidelines through a file-based memory system, achieving rapid performance matching while drastically reducing inference cost.
We propose a framework that amortizes the cost of inference-time reasoning by converting transient critiques into retrievable guidelines, through a file-based memory system and agent-controlled tool calls. We evaluate this method on the Rubric Feedback Bench, a novel dataset for rubric-based learning. Experiments demonstrate that our augmented LLMs rapidly match the performance of test-time refinement pipelines while drastically reducing inference cost.