Enhancing Molecular Property Prediction with Auxiliary Learning and Task-Specific Adaptation
This work addresses generalization issues in molecular property prediction, which is important for drug discovery and materials science, but it is incremental as it builds on existing pretrained GNNs with auxiliary learning techniques.
The paper tackles the problem of poor generalization in pretrained Graph Neural Networks for molecular property prediction by jointly training them with auxiliary tasks and proposing methods like RCGrad to align conflicting gradients, resulting in improvements of up to 7.7% over fine-tuning.
Pretrained Graph Neural Networks have been widely adopted for various molecular property prediction tasks. Despite their ability to encode structural and relational features of molecules, traditional fine-tuning of such pretrained GNNs on the target task can lead to poor generalization. To address this, we explore the adaptation of pretrained GNNs to the target task by jointly training them with multiple auxiliary tasks. This could enable the GNNs to learn both general and task-specific features, which may benefit the target task. However, a major challenge is to determine the relatedness of auxiliary tasks with the target task. To address this, we investigate multiple strategies to measure the relevance of auxiliary tasks and integrate such tasks by adaptively combining task gradients or by learning task weights via bi-level optimization. Additionally, we propose a novel gradient surgery-based approach, Rotation of Conflicting Gradients ($\mathtt{RCGrad}$), that learns to align conflicting auxiliary task gradients through rotation. Our experiments with state-of-the-art pretrained GNNs demonstrate the efficacy of our proposed methods, with improvements of up to 7.7% over fine-tuning. This suggests that incorporating auxiliary tasks along with target task fine-tuning can be an effective way to improve the generalizability of pretrained GNNs for molecular property prediction.