Directional diffusion models for graph representation learning
This work addresses a gap in graph learning for researchers and practitioners by introducing a novel diffusion model variant, though it is incremental as it builds on existing diffusion frameworks.
The paper tackles the problem of applying diffusion models to unsupervised graph representation learning by identifying that isotropic noise in vanilla diffusion processes converts anisotropic graph signals too quickly, hindering meaningful representation acquisition. They propose directional diffusion models with data-dependent anisotropic noise, achieving superior performance over state-of-the-art baselines on 12 datasets for graph representation learning tasks.
In recent years, diffusion models have achieved remarkable success in various domains of artificial intelligence, such as image synthesis, super-resolution, and 3D molecule generation. However, the application of diffusion models in graph learning has received relatively little attention. In this paper, we address this gap by investigating the use of diffusion models for unsupervised graph representation learning. We begin by identifying the anisotropic structures of graphs and a crucial limitation of the vanilla forward diffusion process in learning anisotropic structures. This process relies on continuously adding an isotropic Gaussian noise to the data, which may convert the anisotropic signals to noise too quickly. This rapid conversion hampers the training of denoising neural networks and impedes the acquisition of semantically meaningful representations in the reverse process. To address this challenge, we propose a new class of models called {\it directional diffusion models}. These models incorporate data-dependent, anisotropic, and directional noises in the forward diffusion process. To assess the efficacy of our proposed models, we conduct extensive experiments on 12 publicly available datasets, focusing on two distinct graph representation learning tasks. The experimental results demonstrate the superiority of our models over state-of-the-art baselines, indicating their effectiveness in capturing meaningful graph representations. Our studies not only provide valuable insights into the forward process of diffusion models but also highlight the wide-ranging potential of these models for various graph-related tasks.