Complexity-Aware Deep Symbolic Regression with Robust Risk-Seeking Policy Gradients
This work addresses robustness and efficiency issues in symbolic regression for scientific discovery applications, representing an incremental improvement over existing DSR methods.
The authors tackled the problem of inefficient model updates and manual complexity tuning in deep symbolic regression by developing a decoder-only architecture with frequency-domain attention, a BIC-based reward function, and a ranking-based policy update method, achieving state-of-the-art results on benchmarks with improved robustness and interpretability.
We propose a novel deep symbolic regression approach to enhance the robustness and interpretability of data-driven mathematical expression discovery. Our work is aligned with the popular DSR framework which focuses on learning a data-specific expression generator, without relying on pretrained models or additional search or planning procedures. Despite the success of existing DSR methods, they are built on recurrent neural networks, solely guided by data fitness, and potentially meet tail barriers that can zero out the policy gradient, causing inefficient model updates. To overcome these limitations, we design a decoder-only architecture that performs attention in the frequency domain and introduce a dual-indexed position encoding to conduct layer-wise generation. Second, we propose a Bayesian information criterion (BIC)-based reward function that can automatically adjust the trade-off between expression complexity and data fitness, without the need for explicit manual tuning. Third, we develop a ranking-based weighted policy update method that eliminates the tail barriers and enhances training effectiveness. Extensive benchmarks and systematic experiments demonstrate the advantages of our approach.