SwinGNN: Rethinking Permutation Invariance in Diffusion Models for Graph Generation
This work addresses graph generation for domains like chemistry and biology, offering an incremental improvement by optimizing training and sampling methods to enhance sample quality.
The paper tackles the challenge of graph generation by identifying that permutation-invariant diffusion models face greater learning difficulties due to more complex target distributions, and proposes SwinGNN, a non-invariant model with specific techniques that achieves state-of-the-art performance on synthetic and real-world datasets like proteins and molecules.
Diffusion models based on permutation-equivariant networks can learn permutation-invariant distributions for graph data. However, in comparison to their non-invariant counterparts, we have found that these invariant models encounter greater learning challenges since 1) their effective target distributions exhibit more modes; 2) their optimal one-step denoising scores are the score functions of Gaussian mixtures with more components. Motivated by this analysis, we propose a non-invariant diffusion model, called $\textit{SwinGNN}$, which employs an efficient edge-to-edge 2-WL message passing network and utilizes shifted window based self-attention inspired by SwinTransformers. Further, through systematic ablations, we identify several critical training and sampling techniques that significantly improve the sample quality of graph generation. At last, we introduce a simple post-processing trick, $\textit{i.e.}$, randomly permuting the generated graphs, which provably converts any graph generative model to a permutation-invariant one. Extensive experiments on synthetic and real-world protein and molecule datasets show that our SwinGNN achieves state-of-the-art performances. Our code is released at https://github.com/qiyan98/SwinGNN.