Probabilistic Dual Network Architecture Search on Graphs
This addresses the need for efficient architecture engineering in GNNs, which are crucial for tasks involving non-Euclidean graph data, but the approach is incremental as it builds on existing NAS methods by extending them to graphs.
The paper tackles the problem of automating architecture design for Graph Neural Networks (GNNs) by proposing PDNAS, a differentiable Network Architecture Search method that optimizes both micro- and macro-architectures, resulting in performance gains such as beating competitors by 1.67 and 0.17 in F1 scores on the PPI dataset.
We present the first differentiable Network Architecture Search (NAS) for Graph Neural Networks (GNNs). GNNs show promising performance on a wide range of tasks, but require a large amount of architecture engineering. First, graphs are inherently a non-Euclidean and sophisticated data structure, leading to poor adaptivity of GNN architectures across different datasets. Second, a typical graph block contains numerous different components, such as aggregation and attention, generating a large combinatorial search space. To counter these problems, we propose a Probabilistic Dual Network Architecture Search (PDNAS) framework for GNNs. PDNAS not only optimises the operations within a single graph block (micro-architecture), but also considers how these blocks should be connected to each other (macro-architecture). The dual architecture (micro- and marco-architectures) optimisation allows PDNAS to find deeper GNNs on diverse datasets with better performance compared to other graph NAS methods. Moreover, we use a fully gradient-based search approach to update architectural parameters, making it the first differentiable graph NAS method. PDNAS outperforms existing hand-designed GNNs and NAS results, for example, on the PPI dataset, PDNAS beats its best competitors by 1.67 and 0.17 in F1 scores.