Generating Hadamard matrices with transformers
This work provides a new approach for constructing Hadamard matrices, which are important in combinatorics and signal processing, but the results are incremental as they extend existing techniques.
The paper introduces a transformer-based method within the PatternBoost framework to construct Hadamard matrices, achieving success for orders up to 244 and outperforming random initialization in hard cases.
We present a new method for constructing Hadamard matrices that combines transformer neural networks with local search in the PatternBoost framework. Our approach is designed for extremely sparse combinatorial search problems and is particularly effective for Hadamard matrices of Goethals--Seidel type, where Fourier methods permit fast scoring and optimisation. For orders between $100$ and $250$, it produces large numbers of inequivalent Hadamard matrices, and in harder cases it succeeds where local search from random initialisation fails. The largest example found by our method has order $244$. In addition to these new constructions, our experiments reveal that the transformer can discover and exploit useful hidden symmetry in the search space.