MetaSymNet: A Tree-like Symbol Network with Adaptive Architecture and Activation Functions
This work addresses the challenge of interpretability and efficiency in neural networks for scientific AI, offering a novel approach to symbolic regression, though it appears incremental in the context of existing symbolic methods.
The authors tackled the problem of uninterpretable black-box neural networks and structural redundancy by proposing MetaSymNet, a neural network that dynamically adjusts its architecture and uses an adaptive activation function to evolve into concise mathematical expressions. Experimental results on over 10 datasets with 222 formulas showed that MetaSymNet outperformed four state-of-the-art symbolic regression algorithms consistently, regardless of noise, and excelled in fitting and extrapolation compared to MLP and SVM, with significantly lower network complexity under the same fit.
Mathematical formulas serve as the means of communication between humans and nature, encapsulating the operational laws governing natural phenomena. The concise formulation of these laws is a crucial objective in scientific research and an important challenge for artificial intelligence (AI). While traditional artificial neural networks (MLP) excel at data fitting, they often yield uninterpretable black box results that hinder our understanding of the relationship between variables x and predicted values y. Moreover, the fixed network architecture in MLP often gives rise to redundancy in both network structure and parameters. To address these issues, we propose MetaSymNet, a novel neural network that dynamically adjusts its structure in real-time, allowing for both expansion and contraction. This adaptive network employs the PANGU meta function as its activation function, which is a unique type capable of evolving into various basic functions during training to compose mathematical formulas tailored to specific needs. We then evolve the neural network into a concise, interpretable mathematical expression. To evaluate MetaSymNet's performance, we compare it with four state-of-the-art symbolic regression algorithms across more than 10 public datasets comprising 222 formulas. Our experimental results demonstrate that our algorithm outperforms others consistently regardless of noise presence or absence. Furthermore, we assess MetaSymNet against MLP and SVM regarding their fitting ability and extrapolation capability, these are two essential aspects of machine learning algorithms. The findings reveal that our algorithm excels in both areas. Finally, we compared MetaSymNet with MLP using iterative pruning in network structure complexity. The results show that MetaSymNet's network structure complexity is obviously less than MLP under the same goodness of fit.