A Comparative Review of RNA Language Models
This work addresses the lack of standardized comparisons for RNA language models, which is important for researchers in computational biology and bioinformatics.
The authors conducted a comparative review of 13 RNA language models, along with control models, to assess their performance in zero-shot prediction of RNA secondary structure and functional classification, finding that models excelling in one task often performed poorly in the other.
Given usefulness of protein language models (LMs) in structure and functional inference, RNA LMs have received increased attentions in the last few years. However, these RNA models are often not compared against the same standard. Here, we divided RNA LMs into three classes (pretrained on multiple RNA types (especially noncoding RNAs), specific-purpose RNAs, and LMs that unify RNA with DNA or proteins or both) and compared 13 RNA LMs along with 3 DNA and 1 protein LMs as controls in zero-shot prediction of RNA secondary structure and functional classification. Results shows that the models doing well on secondary structure prediction often perform worse in function classification or vice versa, suggesting that more balanced unsupervised training is needed.