Iterative Neural Networks with Bounded Weights
This work provides theoretical guarantees for neural network stability, but it is incremental as it builds on prior analysis in Hilbert spaces.
The authors tackled the problem of ensuring convergence to a unique fixed point in iterative neural networks by imposing a mild condition on weights, resulting in a bound on the fixed point norm in terms of network parameters.
A recent analysis of a model of iterative neural network in Hilbert spaces established fundamental properties of such networks, such as existence of the fixed points sets, convergence analysis, and Lipschitz continuity. Building on these results, we show that under a single mild condition on the weights of the network, one is guaranteed to obtain a neural network converging to its unique fixed point. We provide a bound on the norm of this fixed point in terms of norms of weights and biases of the network. We also show why this model of a feed-forward neural network is not able to accomodate Hopfield networks under our assumption.