CLEAR: Contrasting Textual Feedback with Experts and Amateurs for Reasoning
This addresses the challenge of enhancing reasoning capabilities in language models for tasks like story generation and math, though it appears incremental as it builds on existing feedback methods.
The paper tackles the problem of improving language model reasoning by introducing CLEAR, which contrasts feedback from expert and amateur models to iteratively refine responses, resulting in performance gains such as up to 19.6% increase in interestingness for story outlines and up to 6.7% improvement in accuracy in mathematical reasoning.
We introduce CLEAR (Contrasting Textual Feedback with Experts and Amateurs for Reasoning), a novel approach to language model reasoning that leverages the strengths of a larger (expert) model and smaller (amateur) model. The expert and amateur models each provide feedback on a model's initial output and are contrasted with each other into refined feedback. This feedback is subsequently applied to iteratively improve CLEAR's responses. Our experiments demonstrate that CLEAR outperforms state-of-the-art methods in several challenging reasoning tasks, including story outline improvement (up to 19.6% relative increase in interestingness), constrained generation (up to 18.5% increase in coverage), mathematical reasoning (up to 6.7% improvement in accuracy) and mitigation of toxicity (decrease of up to 22% in toxicity).