Exploring the Alignment of Generation and Understanding in Protein Structure Modeling
For researchers in protein modeling, this work demonstrates a method to bridge the gap between generation and understanding, improving functional protein generation.
The paper investigates the performance gap between generative and understanding models in protein structure modeling, finding that generative models underperform on understanding tasks. By aligning generative diffusion models with pretrained understanding models via representation alignment, they achieve a 20% relative improvement in functional protein generation on MotifBench (score from 39.2 to 47.1).
Understanding and generation are often treated as two separate paradigms in training deep neural networks, despite the fact that both are trained with closely related objectives such as denoising and masked prediction. While prior studies have shown that generative models often learn suboptimal representations for understanding tasks in vision, it is less understood whether a similar gap exists in the protein domain. In this work, we systematically investigate this question by benchmarking state-of-the-art protein generative models on widely-used protein understanding tasks, and observe that these models exhibit consistently poor performance compared to existing protein encoders. Furthermore, inspired by the Representation Alignment (REPA) framework, we propose to explicitly align generative protein diffusion models with pretrained protein understanding models during training. Experiments on the MotifBench demonstrate that representation alignment significantly improves functional protein generation, boosting the MotifBench score of Protpardelle-1c from 39.2 to 47.1, corresponding to a 20% relative improvement. Our results suggest that representation alignment provides a general and effective mechanism for bridging understanding and generation in protein structure modeling.