Regularizing Semi-supervised Graph Convolutional Networks with a Manifold Smoothness Loss
This addresses overfitting in semi-supervised learning on graphs, but it is incremental as it builds on existing graph networks.
The paper tackles overfitting in semi-supervised graph convolutional networks by proposing an unsupervised manifold smoothness loss as regularization, and shows that adding this loss consistently improves performance in experiments.
Existing graph convolutional networks focus on the neighborhood aggregation scheme. When applied to semi-supervised learning, they often suffer from the overfitting problem as the networks are trained with the cross-entropy loss on a small potion of labeled data. In this paper, we propose an unsupervised manifold smoothness loss defined with respect to the graph structure, which can be added to the loss function as a regularization. We draw connections between the proposed loss with an iterative diffusion process, and show that minimizing the loss is equivalent to aggregate neighbor predictions with infinite layers. We conduct experiments on multi-layer perceptron and existing graph networks, and demonstrate that adding the proposed loss can improve the performance consistently.