Feedback Descent: Open-Ended Text Optimization via Pairwise Comparison
It addresses the bottleneck in preference learning for text optimization, enabling task-agnostic improvements without weight updates, though it is incremental as it builds on existing feedback and in-context learning approaches.
The paper tackles the problem of optimizing text artifacts like prompts, code, and molecules by using structured textual feedback instead of scalar rewards, and it outperforms state-of-the-art methods, achieving novel drug-like molecules above the 99.9th percentile on a benchmark with over 260,000 compounds.
We introduce \textit{Feedback Descent}, a framework that optimizes text artifacts -- prompts, code, and molecules -- through structured textual feedback, rather than relying solely on scalar rewards. By preserving detailed critiques instead of compressing them to binary preferences, Feedback Descent widens the information bottleneck in preference learning, enabling directed optimization in text space rather than weight space. We show that in-context learning can transform structured feedback into gradient-like directional information, enabling targeted edits. Unlike prior approaches that collapse judgments into single bits, our evaluators pair each comparison with textual feedback, which functions as high-bandwidth supervision. The iteration loop is done purely at inference time, without modifying any model weights, and is task-agnostic. We evaluate Feedback Descent on three diverse domains and find that it outperforms state-of-the-art prompt optimization (GEPA), reinforcement learning methods (GRPO, REINVENT), and even specialized graph-based molecular optimizers. In the DOCKSTRING molecule discovery benchmark, Feedback Descent identifies novel drug-like molecules surpassing the $99.9$th percentile of a database with more than $260{,}000$ compounds across six protein targets.