Random Projection Forest Initialization for Graph Convolutional Networks
This work addresses a domain-specific bottleneck in graph-based deep learning by improving graph construction for GCNs, though it is incremental as it builds on existing initialization methods.
The paper tackles the problem of constructing effective graphs for Graph Convolutional Networks (GCNs) by proposing random projection forest (rpForest) initialization, which assigns varying edge weights to enhance learning and outperforms k-nearest neighbor initialization in experiments.
Graph convolutional networks (GCNs) were a great step towards extending deep learning to unstructured data such as graphs. But GCNs still need a constructed graph to work with. To solve this problem, classical graphs such as $k$-nearest neighbor are usually used to initialize the GCN. Although it is computationally efficient to construct $k$-nn graphs, the constructed graph might not be very useful for learning. In a $k$-nn graph, points are restricted to have a fixed number of edges, and all edges in the graph have equal weights. We present a new way to construct the graph and initialize the GCN. It is based on random projection forest (rpForest). rpForest enables us to assign varying weights on edges indicating varying importance, which enhanced the learning. The number of trees is a hyperparameter in rpForest. We performed spectral analysis to help us setting this parameter in the right range. In the experiments, initializing the GCN using rpForest provides better results compared to $k$-nn initialization.