Gransformer: Transformer-based Graph Generation
This work addresses graph generation for applications in network analysis and data synthesis, but it is incremental as it adapts existing Transformer architectures without introducing a fundamentally new paradigm.
The paper tackles the problem of graph generation by proposing Gransformer, a Transformer-based algorithm that modifies the encoder to incorporate structural information and uses a graph-based familiarity measure, achieving performance comparable to existing methods like recurrent models and graph convolutional networks on synthetic and real-world datasets.
Transformers have become widely used in various tasks, such as natural language processing and machine vision. This paper proposes Gransformer, an algorithm based on Transformer for generating graphs. We modify the Transformer encoder to exploit the structural information of the given graph. The attention mechanism is adapted to consider the presence or absence of edges between each pair of nodes. We also introduce a graph-based familiarity measure between node pairs that applies to both the attention and the positional encoding. This measure of familiarity is based on message-passing algorithms and contains structural information about the graph. Also, this measure is autoregressive, which allows our model to acquire the necessary conditional probabilities in a single forward pass. In the output layer, we also use a masked autoencoder for density estimation to efficiently model the sequential generation of dependent edges connected to each node. In addition, we propose a technique to prevent the model from generating isolated nodes without connection to preceding nodes by using BFS node orderings. We evaluate this method using synthetic and real-world datasets and compare it with related ones, including recurrent models and graph convolutional networks. Experimental results show that the proposed method performs comparatively to these methods.