Deliberate Evolution: Agentic Reasoning for Sample-Efficient Symbolic Regression with LLMs
For researchers in symbolic regression and scientific discovery, DE addresses the sample inefficiency of LLM-based evolutionary methods, offering a more efficient approach.
Deliberate Evolution (DE) improves sample efficiency in LLM-based symbolic regression by decoupling candidate generation from search guidance, achieving better performance than baselines while using only 40% of the standard sample budget.
Symbolic regression (SR) discovers compact mathematical expressions from data, yet recent LLM-based evolutionary methods remain sample-inefficient because they rely mainly on scalar feedback such as MSE. We identify a core limitation: existing methods conflate candidate proposal with search guidance, requiring the LLM to infer how to evolve an expression, diagnose its errors, and reuse past experience from a single score. To address this, we propose Deliberate Evolution (DE), an agentic framework that decouples symbolic generation from search control. DE guides LLM proposals with adaptive operators for search direction, analytical tools for structural diagnosis, and reflective memory for trajectory-level experience. Experiments on LLM-SRBench show that DE consistently outperforms representative LLM-based SR baselines across diverse scientific domains while using only 40% of the standard sample budget.