AGNN: Alternating Graph-Regularized Neural Networks to Alleviate Over-Smoothing
This addresses the over-smoothing issue in graph neural networks for researchers and practitioners working with complex graph datasets, representing an incremental improvement over existing methods.
The paper tackles the over-smoothing problem in Graph Convolutional Networks (GCNs) by proposing an Alternating Graph-regularized Neural Network (AGNN), which integrates Graph Convolutional Layers and Graph Embedding Layers to generate more distinguishable features and uses an improved Adaboost strategy for aggregation, resulting in superior performance compared to state-of-the-art models.
Graph Convolutional Network (GCN) with the powerful capacity to explore graph-structural data has gained noticeable success in recent years. Nonetheless, most of the existing GCN-based models suffer from the notorious over-smoothing issue, owing to which shallow networks are extensively adopted. This may be problematic for complex graph datasets because a deeper GCN should be beneficial to propagating information across remote neighbors. Recent works have devoted effort to addressing over-smoothing problems, including establishing residual connection structure or fusing predictions from multi-layer models. Because of the indistinguishable embeddings from deep layers, it is reasonable to generate more reliable predictions before conducting the combination of outputs from various layers. In light of this, we propose an Alternating Graph-regularized Neural Network (AGNN) composed of Graph Convolutional Layer (GCL) and Graph Embedding Layer (GEL). GEL is derived from the graph-regularized optimization containing Laplacian embedding term, which can alleviate the over-smoothing problem by periodic projection from the low-order feature space onto the high-order space. With more distinguishable features of distinct layers, an improved Adaboost strategy is utilized to aggregate outputs from each layer, which explores integrated embeddings of multi-hop neighbors. The proposed model is evaluated via a large number of experiments including performance comparison with some multi-layer or multi-order graph neural networks, which reveals the superior performance improvement of AGNN compared with state-of-the-art models.