GRAN is superior to GraphRNN: node orderings, kernel- and graph embeddings-based metrics for graph generators
This work provides guidelines for selecting and evaluating graph generative models in applications like drug discovery and neural architecture search, though it is incremental as it builds on existing models and metrics.
The study tackled the problem of evaluating graph generative models by comparing kernel- and manifold-based metrics, finding that manifold-based metrics in embedding space outperform kernel-based ones. It demonstrated GRAN's superiority over GraphRNN and showed that adapting GraphRNN with a depth-first search ordering improves performance for small graphs.
A wide variety of generative models for graphs have been proposed. They are used in drug discovery, road networks, neural architecture search, and program synthesis. Generating graphs has theoretical challenges, such as isomorphic representations -- evaluating how well a generative model performs is difficult. Which model to choose depending on the application domain? We extensively study kernel-based metrics on distributions of graph invariants and manifold-based and kernel-based metrics in graph embedding space. Manifold-based metrics outperform kernel-based metrics in embedding space. We use these metrics to compare GraphRNN and GRAN, two well-known generative models for graphs, and unveil the influence of node orderings. It shows the superiority of GRAN over GraphRNN - further, our proposed adaptation of GraphRNN with a depth-first search ordering is effective for small-sized graphs. A guideline on good practices regarding dataset selection and node feature initialization is provided. Our work is accompanied by open-source code and reproducible experiments.