Boolformer: Symbolic Regression of Logic Functions with Transformers
This provides an interpretable alternative to classic machine learning methods for binary classification and gene regulatory network modeling, though it appears incremental as it applies existing Transformer architectures to a specific domain.
The authors tackled symbolic regression of Boolean functions by introducing Boolformer, a Transformer-based model that predicts compact formulas from truth tables, even with incomplete or noisy data, achieving competitive performance with state-of-the-art genetic algorithms while being orders of magnitude faster.
We introduce Boolformer, a Transformer-based model trained to perform end-to-end symbolic regression of Boolean functions. First, we show that it can predict compact formulas for complex functions not seen during training, given their full truth table. Then, we demonstrate that even with incomplete or noisy observations, Boolformer is still able to find good approximate expressions. We evaluate Boolformer on a broad set of real-world binary classification datasets, demonstrating its potential as an interpretable alternative to classic machine learning methods. Finally, we apply it to the widespread task of modeling the dynamics of gene regulatory networks and show through a benchmark that Boolformer is competitive with state-of-the-art genetic algorithms, with a speedup of several orders of magnitude. Our code and models are available publicly.