Programmatic Context Augmentation for LLM-based Symbolic Regression
This work addresses the limitation of existing LLM-based symbolic regression methods that rely solely on scalar feedback, offering a more effective approach for scientific discovery tasks.
The paper introduces a novel LLM-based evolutionary search framework for symbolic regression that uses programmatic context augmentation to extract richer information from datasets beyond scalar metrics, achieving superior efficiency and accuracy on LLM-SRBench benchmarks.
Symbolic regression (SR), the task of discovering mathematical expressions that best describe a given dataset, remains a fundamental challenge in scientific discovery. Traditional approaches, primarily based on genetic algorithms and related evolutionary methods, have proven useful but suffer from scalability and expressivity limitations. Recently, large language model (LLM)-based evolutionary search methods have been introduced into SR and show promise. However, existing LLM-based approaches typically rely on scalar evaluation metrics, such as mean squared error, as the sole source of feedback during the search process, thereby overlooking the rich information embedded in the dataset. To address this limitation, we propose a novel LLM-based evolutionary search framework that incorporates programmatic context augmentation. By enabling code-based interactions with the dataset, our method can actively perform data analysis and extract informative signals, beyond aggregated evaluation scores. We evaluate our framework on advanced benchmarks, such as LLM-SRBench, and demonstrate superior efficiency and accuracy compared to strong baselines.