Prompt-Guided Injection of Conformation to Pre-trained Protein Model
This work addresses the problem of making protein models more adaptable for various biological tasks, such as protein folding and interactions, for researchers in computational biology, though it is incremental by building on existing PTPM frameworks.
The paper tackles the limitation of pre-trained protein models (PTPMs) in handling diverse tasks due to fixed embeddings, by proposing interpretable prompts to inject task-related knowledge, such as conformational information for protein folding. Results show that incorporating an interaction-conformation prompt significantly improves performance on tasks requiring conformational knowledge across nine protein datasets, without harming sequence-related tasks.
Pre-trained protein models (PTPMs) represent a protein with one fixed embedding and thus are not capable for diverse tasks. For example, protein structures can shift, namely protein folding, between several conformations in various biological processes. To enable PTPMs to produce task-aware representations, we propose to learn interpretable, pluggable and extensible protein prompts as a way of injecting task-related knowledge into PTPMs. In this regard, prior PTPM optimization with the masked language modeling task can be interpreted as learning a sequence prompt (Seq prompt) that enables PTPMs to capture the sequential dependency between amino acids. To incorporate conformational knowledge to PTPMs, we propose an interaction-conformation prompt (IC prompt) that is learned through back-propagation with the protein-protein interaction task. As an instantiation, we present a conformation-aware pre-trained protein model that learns both sequence and interaction-conformation prompts in a multi-task setting. We conduct comprehensive experiments on nine protein datasets. Results confirm our expectation that using the sequence prompt does not hurt PTPMs' performance on sequence-related tasks while incorporating the interaction-conformation prompt significantly improves PTPMs' performance on tasks where conformational knowledge counts. We also show the learned prompts can be combined and extended to deal with new complex tasks.