FoldToken: Learning Protein Language via Vector Quantization and Beyond
This work addresses the problem of integrating discrete protein sequences with continuous 3D structures for researchers in computational biology, offering a novel approach for protein design tasks.
The authors tackled the challenge of representing protein sequences and structures in a unified discrete language by introducing FoldTokenizer, which projects residues into discrete symbols called FoldTokens, and applied this to backbone inpainting and antibody design, achieving promising results with their FoldGPT model.
Is there a foreign language describing protein sequences and structures simultaneously? Protein structures, represented by continuous 3D points, have long posed a challenge due to the contrasting modeling paradigms of discrete sequences. We introduce \textbf{FoldTokenizer} to represent protein sequence-structure as discrete symbols. This innovative approach involves projecting residue types and structures into a discrete space, guided by a reconstruction loss for information preservation. We refer to the learned discrete symbols as \textbf{FoldToken}, and the sequence of FoldTokens serves as a new protein language, transforming the protein sequence-structure into a unified modality. We apply the created protein language on general backbone inpainting and antibody design tasks, building the first GPT-style model (\textbf{FoldGPT}) for sequence-structure co-generation with promising results. Key to our success is the substantial enhancement of the vector quantization module, Soft Conditional Vector Quantization (\textbf{SoftCVQ}).