L_1-regularized Boltzmann machine learning using majorizer minimization
This work addresses the challenge of sparse inference in Boltzmann machines for researchers in machine learning, but it is incremental as it adapts an existing optimization technique to a known bottleneck.
The paper tackles the problem of estimating sparse interactions and biases in Boltzmann machine learning by using L1 regularization, which introduces non-smoothness, and applies the majorizer minimization method to overcome this issue, resulting in the ability to elucidate relevant biases and interactions from data with strongly-correlated components.
We propose an inference method to estimate sparse interactions and biases according to Boltzmann machine learning. The basis of this method is $L_1$ regularization, which is often used in compressed sensing, a technique for reconstructing sparse input signals from undersampled outputs. $L_1$ regularization impedes the simple application of the gradient method, which optimizes the cost function that leads to accurate estimations, owing to the cost function's lack of smoothness. In this study, we utilize the majorizer minimization method, which is a well-known technique implemented in optimization problems, to avoid the non-smoothness of the cost function. By using the majorizer minimization method, we elucidate essentially relevant biases and interactions from given data with seemingly strongly-correlated components.