Universal Graph Transformer Self-Attention Networks
This addresses graph learning problems for researchers and practitioners, but it is incremental as it adapts transformers to GNNs.
The paper tackles graph representation learning by introducing UGformer, a transformer-based GNN model with two variants: one using sampled neighbors and another using all nodes, achieving state-of-the-art accuracies on graph classification and inductive text classification benchmarks.
We introduce a transformer-based GNN model, named UGformer, to learn graph representations. In particular, we present two UGformer variants, wherein the first variant (publicized in September 2019) is to leverage the transformer on a set of sampled neighbors for each input node, while the second (publicized in May 2021) is to leverage the transformer on all input nodes. Experimental results demonstrate that the first UGformer variant achieves state-of-the-art accuracies on benchmark datasets for graph classification in both inductive setting and unsupervised transductive setting; and the second UGformer variant obtains state-of-the-art accuracies for inductive text classification. The code is available at: \url{https://github.com/daiquocnguyen/Graph-Transformer}.