Distilling Wikipedia mathematical knowledge into neural network models
This addresses the data scarcity problem for researchers in symbolic mathematics by providing a centralized source of training data, though it is incremental as it adapts existing NLP resources to a new domain.
The authors tackled the lack of real-world symbolic expression data for machine learning by creating a pipeline to extract mathematical expressions from Wikipedia, training a mathematical language model on this corpus, and using it as a prior to enhance neural-guided search for symbolic regression, resulting in improved performance.
Machine learning applications to symbolic mathematics are becoming increasingly popular, yet there lacks a centralized source of real-world symbolic expressions to be used as training data. In contrast, the field of natural language processing leverages resources like Wikipedia that provide enormous amounts of real-world textual data. Adopting the philosophy of "mathematics as language," we bridge this gap by introducing a pipeline for distilling mathematical expressions embedded in Wikipedia into symbolic encodings to be used in downstream machine learning tasks. We demonstrate that a $\textit{mathematical}$ $\textit{language}$ $\textit{model}$ trained on this "corpus" of expressions can be used as a prior to improve the performance of neural-guided search for the task of symbolic regression.