Invariance-Preserving Localized Activation Functions for Graph Neural Networks
This work addresses the need for more expressive activation functions in GNNs for tasks like graph-based classification and recommendation, but it is incremental as it builds on existing GNN frameworks.
The paper tackled the problem of designing trainable nonlinear activation functions for Graph Neural Networks (GNNs) that incorporate graph structure, using graph median and max filters to preserve permutation invariance. The result showed improved model capacity in experiments like source localization and authorship attribution, with localized activation functions outperforming standard ones.
Graph signals are signals with an irregular structure that can be described by a graph. Graph neural networks (GNNs) are information processing architectures tailored to these graph signals and made of stacked layers that compose graph convolutional filters with nonlinear activation functions. Graph convolutions endow GNNs with invariance to permutations of the graph nodes' labels. In this paper, we consider the design of trainable nonlinear activation functions that take into consideration the structure of the graph. This is accomplished by using graph median filters and graph max filters, which mimic linear graph convolutions and are shown to retain the permutation invariance of GNNs. We also discuss modifications to the backpropagation algorithm necessary to train local activation functions. The advantages of localized activation function architectures are demonstrated in four numerical experiments: source localization on synthetic graphs, authorship attribution of 19th century novels, movie recommender systems and scientific article classification. In all cases, localized activation functions are shown to improve model capacity.