Sampling two-dimensional spin systems with transformers
This work addresses the computational inefficiency of transformer-based sampling for spin systems, offering a practical improvement for statistical physics simulations.
The authors propose a transformer-based neural sampler for classical spin systems that generates groups of spins per step and uses approximated probabilities, achieving an Effective Sample Size ~20 times larger than the previous state-of-the-art for the 128x128 Ising model at critical temperature, and scaling to 180x180 systems.
Autoregressive Neural Networks based on dense or convolutional layers have recently been shown to be a viable strategy for generating classical spin systems. Unlike these methods, sampling with transformers is commonly considered to be computationally inefficient. In this work, we propose a novel approach to transformer-based neural samplers in which we generate not a single spin per step but groups of spins. As an additional improvement, we construct a model of approximated probabilities, further improving the efficiency of the algorithm. Despite our approach being computationally heavier than dense networks or CNN-based approaches, we were able to sample larger systems of up to $180 \times 180$ spins in case of the Ising model. The Effective Sample Size of our sampler is $\sim 20$ times larger than that of the previous state-of-the-art neural sampler when trained for the $128 \times 128$ Ising model at critical temperature. Finally, we also test our algorithm on the 2D Edwards-Anderson model, where we train $64\times 64$ spin systems.