Median activation functions for graph neural networks
This addresses a specific problem in graph neural networks for researchers and practitioners by introducing a novel activation function to enhance local nonlinear encoding, though it appears incremental as it builds on existing GNN architectures.
The paper tackled the issue that linear shift-invariant graph filters and pointwise activation functions fail to encode local nonlinear graph signal behavior in graph neural networks (GNNs), and proposed median activation functions with support on graph neighborhoods, showing that they can improve GNN capacity with marginal increase in complexity on synthetic and real-world datasets.
Graph neural networks (GNNs) have been shown to replicate convolutional neural networks' (CNNs) superior performance in many problems involving graphs. By replacing regular convolutions with linear shift-invariant graph filters (LSI-GFs), GNNs take into account the (irregular) structure of the graph and provide meaningful representations of network data. However, LSI-GFs fail to encode local nonlinear graph signal behavior, and so do regular activation functions, which are nonlinear but pointwise. To address this issue, we propose median activation functions with support on graph neighborhoods instead of individual nodes. A GNN architecture with a trainable multirresolution version of this activation function is then tested on synthetic and real-word datasets, where we show that median activation functions can improve GNN capacity with marginal increase in complexity.