Stability of Algebraic Neural Networks to Small Perturbations
This addresses stability issues for researchers and practitioners using neural networks with convolution operators, such as CNNs and GNNs, but appears incremental as it extends existing algebraic frameworks.
The paper tackles the problem of ensuring stability in algebraic neural networks (AlgNNs) to small perturbations, showing that architectures using formal convolution can be stable regardless of specific shift operator choices, with stability depending on algebraic subset structures.
Algebraic neural networks (AlgNNs) are composed of a cascade of layers each one associated to and algebraic signal model, and information is mapped between layers by means of a nonlinearity function. AlgNNs provide a generalization of neural network architectures where formal convolution operators are used, like for instance traditional neural networks (CNNs) and graph neural networks (GNNs). In this paper we study stability of AlgNNs on the framework of algebraic signal processing. We show how any architecture that uses a formal notion of convolution can be stable beyond particular choices of the shift operator, and this stability depends on the structure of subsets of the algebra involved in the model. We focus our attention on the case of algebras with a single generator.