Graph Representation Learning via Multi-task Knowledge Distillation
This work addresses graph-level prediction problems where training labels are limited, offering an incremental improvement by leveraging domain expertise to enhance representation learning.
The paper tackled the problem of graph-level representation learning with scarce training labels by proposing a multi-task knowledge distillation method that incorporates network-theory-based graph metrics as auxiliary tasks, showing improved prediction performance on synthetic and real datasets, especially with small training data sizes.
Machine learning on graph structured data has attracted much research interest due to its ubiquity in real world data. However, how to efficiently represent graph data in a general way is still an open problem. Traditional methods use handcraft graph features in a tabular form but suffer from the defects of domain expertise requirement and information loss. Graph representation learning overcomes these defects by automatically learning the continuous representations from graph structures, but they require abundant training labels, which are often hard to fulfill for graph-level prediction problems. In this work, we demonstrate that, if available, the domain expertise used for designing handcraft graph features can improve the graph-level representation learning when training labels are scarce. Specifically, we proposed a multi-task knowledge distillation method. By incorporating network-theory-based graph metrics as auxiliary tasks, we show on both synthetic and real datasets that the proposed multi-task learning method can improve the prediction performance of the original learning task, especially when the training data size is small.