Recursive Graphical Neural Networks for Text Classification
This addresses a key bottleneck in applying GNNs to text classification, offering a domain-specific improvement for natural language processing tasks.
The paper tackles the over-smoothing problem in Graph Neural Networks for text classification by proposing a Recursive Graphical Neural Networks model that uses LSTM to dynamically filter neighbor information and includes a global graph-level node, achieving significant performance gains over strong baselines on most datasets.
The complicated syntax structure of natural language is hard to be explicitly modeled by sequence-based models. Graph is a natural structure to describe the complicated relation between tokens. The recent advance in Graph Neural Networks (GNN) provides a powerful tool to model graph structure data, but simple graph models such as Graph Convolutional Networks (GCN) suffer from over-smoothing problem, that is, when stacking multiple layers, all nodes will converge to the same value. In this paper, we propose a novel Recursive Graphical Neural Networks model (ReGNN) to represent text organized in the form of graph. In our proposed model, LSTM is used to dynamically decide which part of the aggregated neighbor information should be transmitted to upper layers thus alleviating the over-smoothing problem. Furthermore, to encourage the exchange between the local and global information, a global graph-level node is designed. We conduct experiments on both single and multiple label text classification tasks. Experiment results show that our ReGNN model surpasses the strong baselines significantly in most of the datasets and greatly alleviates the over-smoothing problem.