Univariate Skeleton Prediction in Multivariate Systems Using Transformers
This addresses the challenge of interpretability in multivariate systems for researchers in symbolic regression, though it is incremental as it builds on existing neural and transformer approaches.
The paper tackles the problem of identifying functional forms in multivariate symbolic regression by proposing an explainable neural method that generates univariate symbolic skeletons to explain variable influences, and it outperforms four existing methods in experiments.
Symbolic regression (SR) methods attempt to learn mathematical expressions that approximate the behavior of an observed system. However, when dealing with multivariate systems, they often fail to identify the functional form that explains the relationship between each variable and the system's response. To begin to address this, we propose an explainable neural SR method that generates univariate symbolic skeletons that aim to explain how each variable influences the system's response. By analyzing multiple sets of data generated artificially, where one input variable varies while others are fixed, relationships are modeled separately for each input variable. The response of such artificial data sets is estimated using a regression neural network (NN). Finally, the multiple sets of input-response pairs are processed by a pre-trained Multi-Set Transformer that solves a problem we termed Multi-Set Skeleton Prediction and outputs a univariate symbolic skeleton. Thus, such skeletons represent explanations of the function approximated by the regression NN. Experimental results demonstrate that this method learns skeleton expressions matching the underlying functions and outperforms two GP-based and two neural SR methods.