A SOM-based Gradient-Free Deep Learning Method with Convergence Analysis
This addresses gradient-related issues in deep learning for researchers, but appears incremental as it modifies existing Self-Organizing Maps.
The paper tackles the problems caused by gradient descent in deep learning by proposing a gradient-free method using a Deep Valued Self-Organizing Map network with residual connections, and it proves convergence with an inequality linking parameters, input dimension, and prediction loss.
As gradient descent method in deep learning causes a series of questions, this paper proposes a novel gradient-free deep learning structure. By adding a new module into traditional Self-Organizing Map and introducing residual into the map, a Deep Valued Self-Organizing Map network is constructed. And analysis about the convergence performance of such a deep Valued Self-Organizing Map network is proved in this paper, which gives an inequality about the designed parameters with the dimension of inputs and the loss of prediction.