Ordered GNN: Ordering Message Passing to Deal with Heterophily and Over-smoothing
This addresses key limitations in graph neural networks for applications requiring deep or heterophilous graph learning, though it is an incremental improvement over existing message-passing methods.
The paper tackles the over-smoothing and heterophily problems in graph neural networks by ordering message passing into node representations, achieving state-of-the-art results on homophily and heterophily datasets and maintaining performance in deep models.
Most graph neural networks follow the message passing mechanism. However, it faces the over-smoothing problem when multiple times of message passing is applied to a graph, causing indistinguishable node representations and prevents the model to effectively learn dependencies between farther-away nodes. On the other hand, features of neighboring nodes with different labels are likely to be falsely mixed, resulting in the heterophily problem. In this work, we propose to order the messages passing into the node representation, with specific blocks of neurons targeted for message passing within specific hops. This is achieved by aligning the hierarchy of the rooted-tree of a central node with the ordered neurons in its node representation. Experimental results on an extensive set of datasets show that our model can simultaneously achieve the state-of-the-art in both homophily and heterophily settings, without any targeted design. Moreover, its performance maintains pretty well while the model becomes really deep, effectively preventing the over-smoothing problem. Finally, visualizing the gating vectors shows that our model learns to behave differently between homophily and heterophily settings, providing an explainable graph neural model.