An Exploration of Conditioning Methods in Graph Neural Networks
This work addresses a methodological issue for researchers in graph-based deep learning, but it is incremental as it builds on existing GNN frameworks.
The paper tackled the problem of how to condition on node and edge attributes in graph neural networks (GNNs) to improve performance, and found that different conditioning methods (weak, strong, pure) affect model effectiveness in computational chemistry tasks.
The flexibility and effectiveness of message passing based graph neural networks (GNNs) induced considerable advances in deep learning on graph-structured data. In such approaches, GNNs recursively update node representations based on their neighbors and they gain expressivity through the use of node and edge attribute vectors. E.g., in computational tasks such as physics and chemistry usage of edge attributes such as relative position or distance proved to be essential. In this work, we address not what kind of attributes to use, but how to condition on this information to improve model performance. We consider three types of conditioning; weak, strong, and pure, which respectively relate to concatenation-based conditioning, gating, and transformations that are causally dependent on the attributes. This categorization provides a unifying viewpoint on different classes of GNNs, from separable convolutions to various forms of message passing networks. We provide an empirical study on the effect of conditioning methods in several tasks in computational chemistry.