Graph Representation Learning for Multi-Task Settings: a Meta-Learning Approach
This addresses the challenge for researchers and practitioners in graph machine learning who need versatile embeddings for multi-task applications, though it is incremental as it builds on existing meta-learning and GNN techniques.
The paper tackled the problem of generating node embeddings from Graph Neural Networks (GNNs) that can perform well across multiple tasks, rather than being specialized for single tasks, by using a meta-learning approach. The result showed that embeddings produced by their method achieved comparable or even higher performance than single-task and multi-task end-to-end models in experiments.
Graph Neural Networks (GNNs) have become the state-of-the-art method for many applications on graph structured data. GNNs are a model for graph representation learning, which aims at learning to generate low dimensional node embeddings that encapsulate structural and feature-related information. GNNs are usually trained in an end-to-end fashion, leading to highly specialized node embeddings. While this approach achieves great results in the single-task setting, the generation of node embeddings that can be used to perform multiple tasks (with performance comparable to single-task models) is still an open problem. We propose the use of meta-learning to allow the training of a GNN model capable of producing multi-task node embeddings. In particular, we exploit the properties of optimization-based meta-learning to learn GNNs that can produce general node representations by learning parameters that can quickly (i.e. with a few steps of gradient descent) adapt to multiple tasks. Our experiments show that the embeddings produced by a model trained with our purposely designed meta-learning procedure can be used to perform multiple tasks with comparable or, surprisingly, even higher performance than both single-task and multi-task end-to-end models.