Fixed points of nonnegative neural networks
This work provides foundational mathematical insights for understanding autoencoders and deep equilibrium models, though it is incremental in applying existing theory to a specific network type.
The paper tackles the analysis of nonnegative neural networks using fixed point theory, showing that such networks can be recognized as monotonic and scalable mappings, which leads to weaker conditions for fixed point existence and characterizes the fixed point set as an interval that often reduces to a point.
We use fixed point theory to analyze nonnegative neural networks, which we define as neural networks that map nonnegative vectors to nonnegative vectors. We first show that nonnegative neural networks with nonnegative weights and biases can be recognized as monotonic and (weakly) scalable mappings within the framework of nonlinear Perron-Frobenius theory. This fact enables us to provide conditions for the existence of fixed points of nonnegative neural networks having inputs and outputs of the same dimension, and these conditions are weaker than those recently obtained using arguments in convex analysis. Furthermore, we prove that the shape of the fixed point set of nonnegative neural networks with nonnegative weights and biases is an interval, which under mild conditions degenerates to a point. These results are then used to obtain the existence of fixed points of more general nonnegative neural networks. From a practical perspective, our results contribute to the understanding of the behavior of autoencoders, and we also offer valuable mathematical machinery for future developments in deep equilibrium models.