A Phylogenetic Approach to Genomic Language Modeling
This work addresses the challenge of improving genomic variant prediction for researchers in genomics and bioinformatics, representing an incremental advancement.
The authors tackled the problem of genomic language models having limited success in identifying evolutionarily constrained elements by introducing a framework that models nucleotide evolution on phylogenetic trees, resulting in PhyloGPN, which excels at predicting functionally disruptive variants from a single sequence and shows strong transfer learning capabilities.
Genomic language models (gLMs) have shown mostly modest success in identifying evolutionarily constrained elements in mammalian genomes. To address this issue, we introduce a novel framework for training gLMs that explicitly models nucleotide evolution on phylogenetic trees using multispecies whole-genome alignments. Our approach integrates an alignment into the loss function during training but does not require it for making predictions, thereby enhancing the model's applicability. We applied this framework to train PhyloGPN, a model that excels at predicting functionally disruptive variants from a single sequence alone and demonstrates strong transfer learning capabilities.