Performance Evaluation of Ising and QUBO Variable Encodings in Boltzmann Machine Learning
This work provides practical guidelines for variable encoding in Boltzmann machines, addressing a domain-specific issue for researchers in machine learning and optimization.
The study compared Ising and QUBO encodings in Boltzmann machine learning, finding that QUBO leads to slower convergence under stochastic gradient descent due to ill-conditioning, while natural gradient descent achieves similar performance across encodings.
We compare Ising ({-1,+1}) and QUBO ({0,1}) encodings for Boltzmann machine learning under a controlled protocol that fixes the model, sampler, and step size. Exploiting the identity that the Fisher information matrix (FIM) equals the covariance of sufficient statistics, we visualize empirical moments from model samples and reveal systematic, representation-dependent differences. QUBO induces larger cross terms between first- and second-order statistics, creating more small-eigenvalue directions in the FIM and lowering spectral entropy. This ill-conditioning explains slower convergence under stochastic gradient descent (SGD). In contrast, natural gradient descent (NGD)-which rescales updates by the FIM metric-achieves similar convergence across encodings due to reparameterization invariance. Practically, for SGD-based training, the Ising encoding provides more isotropic curvature and faster convergence; for QUBO, centering/scaling or NGD-style preconditioning mitigates curvature pathologies. These results clarify how representation shapes information geometry and finite-time learning dynamics in Boltzmann machines and yield actionable guidelines for variable encoding and preprocessing.