Robust Graph Representation Learning via Predictive Coding
This addresses robustness issues in graph neural networks for machine learning applications, but it is incremental as it adapts an existing framework to a specific domain.
The paper tackled the vulnerability of graph neural networks to adversarial attacks and poor out-of-distribution generalization by using predictive coding message-passing rules, resulting in models that are comparable in performance, better calibrated, and robust against multiple adversarial attacks.
Predictive coding is a message-passing framework initially developed to model information processing in the brain, and now also topic of research in machine learning due to some interesting properties. One of such properties is the natural ability of generative models to learn robust representations thanks to their peculiar credit assignment rule, that allows neural activities to converge to a solution before updating the synaptic weights. Graph neural networks are also message-passing models, which have recently shown outstanding results in diverse types of tasks in machine learning, providing interdisciplinary state-of-the-art performance on structured data. However, they are vulnerable to imperceptible adversarial attacks, and unfit for out-of-distribution generalization. In this work, we address this by building models that have the same structure of popular graph neural network architectures, but rely on the message-passing rule of predictive coding. Through an extensive set of experiments, we show that the proposed models are (i) comparable to standard ones in terms of performance in both inductive and transductive tasks, (ii) better calibrated, and (iii) robust against multiple kinds of adversarial attacks.