Unsupervised language models for disease variant prediction
This addresses the challenge of sparse labels in disease variant prediction for biomedical researchers, offering a more scalable and efficient approach.
The authors tackled the problem of predicting pathogenicity of protein variants by proposing VELM, an unsupervised method that uses a single pretrained protein language model to score variants zero-shot without multiple sequence alignments or per-gene training, achieving performance comparable to state-of-the-art methods on clinically labeled variants.
There is considerable interest in predicting the pathogenicity of protein variants in human genes. Due to the sparsity of high quality labels, recent approaches turn to \textit{unsupervised} learning, using Multiple Sequence Alignments (MSAs) to train generative models of natural sequence variation within each gene. These generative models then predict variant likelihood as a proxy to evolutionary fitness. In this work we instead combine this evolutionary principle with pretrained protein language models (LMs), which have already shown promising results in predicting protein structure and function. Instead of training separate models per-gene, we find that a single protein LM trained on broad sequence datasets can score pathogenicity for any gene variant zero-shot, without MSAs or finetuning. We call this unsupervised approach \textbf{VELM} (Variant Effect via Language Models), and show that it achieves scoring performance comparable to the state of the art when evaluated on clinically labeled variants of disease-related genes.