Deep Generative Symbolic Regression with Monte-Carlo-Tree-Search
This addresses the challenge of low performance in neural symbolic regression methods for out-of-distribution data, representing an incremental improvement over existing approaches.
The paper tackles the problem of symbolic regression by combining neural models with search abilities to improve performance on out-of-distribution datasets, achieving state-of-the-art results on the SRBench benchmark.
Symbolic regression (SR) is the problem of learning a symbolic expression from numerical data. Recently, deep neural models trained on procedurally-generated synthetic datasets showed competitive performance compared to more classical Genetic Programming (GP) algorithms. Unlike their GP counterparts, these neural approaches are trained to generate expressions from datasets given as context. This allows them to produce accurate expressions in a single forward pass at test time. However, they usually do not benefit from search abilities, which result in low performance compared to GP on out-of-distribution datasets. In this paper, we propose a novel method which provides the best of both worlds, based on a Monte-Carlo Tree Search procedure using a context-aware neural mutation model, which is initially pre-trained to learn promising mutations, and further refined from successful experiences in an online fashion. The approach demonstrates state-of-the-art performance on the well-known \texttt{SRBench} benchmark.