On the Effectiveness of Random Weights in Graph Neural Networks
This offers a lightweight and efficient alternative for GNN design, potentially benefiting researchers and practitioners in graph-based machine learning, though it is incremental in nature.
The paper tackles the problem of reducing computational costs in Graph Neural Networks (GNNs) by showing that using random weights instead of learned weights achieves comparable performance across tasks and datasets, with up to 6x faster training and 3x lower memory usage.
Graph Neural Networks (GNNs) have achieved remarkable success across diverse tasks on graph-structured data, primarily through the use of learned weights in message passing layers. In this paper, we demonstrate that random weights can be surprisingly effective, achieving performance comparable to end-to-end training counterparts, across various tasks and datasets. Specifically, we show that by replacing learnable weights with random weights, GNNs can retain strong predictive power, while significantly reducing training time by up to 6$\times$ and memory usage by up to 3$\times$. Moreover, the random weights combined with our construction yield random graph propagation operators, which we show to reduce the problem of feature rank collapse in GNNs. These understandings and empirical results highlight random weights as a lightweight and efficient alternative, offering a compelling perspective on the design and training of GNN architectures.