Subgraph-level Universal Prompt Tuning
This work addresses the problem of limited generalizability in prompt tuning for graph neural networks across different pre-training strategies, offering a more versatile solution for downstream graph applications, though it appears incremental as it builds on existing simple prompt tuning methods.
The paper tackles the challenge of adapting pre-trained graph neural networks via prompt tuning by introducing Subgraph-level Universal Prompt Tuning (SUPT), which assigns prompt features at the subgraph-level to better capture graph contexts, resulting in outperforming fine-tuning-based methods in 42 out of 45 full-shot experiments with over 2.5% average improvement and in 41 out of 45 few-shot experiments with over 6.6% average increase.
In the evolving landscape of machine learning, the adaptation of pre-trained models through prompt tuning has become increasingly prominent. This trend is particularly observable in the graph domain, where diverse pre-training strategies present unique challenges in developing effective prompt-based tuning methods for graph neural networks. Previous approaches have been limited, focusing on specialized prompting functions tailored to models with edge prediction pre-training tasks. These methods, however, suffer from a lack of generalizability across different pre-training strategies. Recently, a simple prompt tuning method has been designed for any pre-training strategy, functioning within the input graph's feature space. This allows it to theoretically emulate any type of prompting function, thereby significantly increasing its versatility for a range of downstream applications. Nevertheless, the capacity of such simple prompts to fully grasp the complex contexts found in graphs remains an open question, necessitating further investigation. Addressing this challenge, our work introduces the Subgraph-level Universal Prompt Tuning (SUPT) approach, focusing on the detailed context within subgraphs. In SUPT, prompt features are assigned at the subgraph-level, preserving the method's universal capability. This requires extremely fewer tuning parameters than fine-tuning-based methods, outperforming them in 42 out of 45 full-shot scenario experiments with an average improvement of over 2.5%. In few-shot scenarios, it excels in 41 out of 45 experiments, achieving an average performance increase of more than 6.6%.