Supervised Attention Using Homophily in Graph Neural Networks
This work addresses a specific issue in graph neural networks for node classification tasks, offering an incremental improvement to existing models.
The paper tackles the problem of graph attention networks (GATs) producing poorly separated node representations due to aggregating messages from nodes of different classes, which can hurt performance. The authors propose a technique to encourage higher attention scores between nodes with the same class label, demonstrating increased performance on several node classification datasets.
Graph neural networks have become the standard approach for dealing with learning problems on graphs. Among the different variants of graph neural networks, graph attention networks (GATs) have been applied with great success to different tasks. In the GAT model, each node assigns an importance score to its neighbors using an attention mechanism. However, similar to other graph neural networks, GATs aggregate messages from nodes that belong to different classes, and therefore produce node representations that are not well separated with respect to the different classes, which might hurt their performance. In this work, to alleviate this problem, we propose a new technique that can be incorporated into any graph attention model to encourage higher attention scores between nodes that share the same class label. We evaluate the proposed method on several node classification datasets demonstrating increased performance over standard baseline models.