Learning Graph Pooling and Hybrid Convolutional Operations for Text Representations
This work addresses graph-based text representation problems for researchers and practitioners in natural language processing, offering incremental improvements with novel layers.
The paper tackles the lack of effective pooling methods for graphs and the limitation of GCNs in handling node order information for text representation, proposing graph pooling (gPool) and hybrid convolutional (hConv) layers that achieve new state-of-the-art performance in text categorization tasks.
With the development of graph convolutional networks (GCN), deep learning methods have started to be used on graph data. In additional to convolutional layers, pooling layers are another important components of deep learning. However, no effective pooling methods have been developed for graphs currently. In this work, we propose the graph pooling (gPool) layer, which employs a trainable projection vector to measure the importance of nodes in graphs. By selecting the k-most important nodes to form the new graph, gPool achieves the same objective as regular max pooling layers operating on images. Another limitation of GCN when used on graph-based text representation tasks is that, GCNs do not consider the order information of nodes in graph. To address this limitation, we propose the hybrid convolutional (hConv) layer that combines GCN and regular convolutional operations. The hConv layer is capable of increasing receptive fields quickly and computing features automatically. Based on the proposed gPool and hConv layers, we develop new deep networks for text categorization tasks. Our results show that the networks based on gPool and hConv layers achieves new state-of-the-art performance as compared to baseline methods.