Learn Layer-wise Connections in Graph Neural Networks
This work addresses the need for adaptive layer-wise connections in GNNs to handle diverse graph types, offering a domain-specific solution that is incremental in nature.
The paper tackles the problem of data-specific layer-wise connections in Graph Neural Networks (GNNs) by proposing a novel framework called LLC that uses neural architecture search to learn adaptive connections, resulting in improved performance and alleviation of over-smoothing across five real-world datasets.
In recent years, Graph Neural Networks (GNNs) have shown superior performance on diverse applications on real-world datasets. To improve the model capacity and alleviate the over-smoothing problem, several methods proposed to incorporate the intermediate layers by layer-wise connections. However, due to the highly diverse graph types, the performance of existing methods vary on diverse graphs, leading to a need for data-specific layer-wise connection methods. To address this problem, we propose a novel framework LLC (Learn Layer-wise Connections) based on neural architecture search (NAS) to learn adaptive connections among intermediate layers in GNNs. LLC contains one novel search space which consists of 3 types of blocks and learnable connections, and one differentiable search algorithm to enable the efficient search process. Extensive experiments on five real-world datasets are conducted, and the results show that the searched layer-wise connections can not only improve the performance but also alleviate the over-smoothing problem.