Steering Protein Language Models
This work addresses the challenge of precise protein engineering for researchers and practitioners, though it is incremental as it adapts existing techniques from text models to proteins.
The paper tackled the problem of controlling Protein Language Models (PLMs) to generate proteins with specific functionalities, demonstrating that Activation Steering can effectively steer PLMs for targeted sequence generation and optimization without additional training.
Protein Language Models (PLMs), pre-trained on extensive evolutionary data from natural proteins, have emerged as indispensable tools for protein design. While powerful, PLMs often struggle to produce proteins with precisely specified functionalities or properties due to inherent challenges in controlling their outputs. In this work, we investigate the potential of Activation Steering, a technique originally developed for controlling text generation in Large Language Models (LLMs), to direct PLMs toward generating protein sequences with targeted properties. We propose a simple yet effective method that employs activation editing to steer PLM outputs, and extend this approach to protein optimization through a novel editing site identification module. Through comprehensive experiments on lysozyme-like sequence generation and optimization, we demonstrate that our methods can be seamlessly integrated into both auto-encoding and autoregressive PLMs without requiring additional training. These results highlight a promising direction for precise protein engineering using foundation models.