Towards Applying Large Language Models to Complement Single-Cell Foundation Models
This work addresses the problem of integrating textual biological information with single-cell omics data for researchers in computational biology, representing an incremental improvement through model fusion.
The authors investigated how large language models (LLMs) can complement single-cell foundation models like scGPT by identifying biological insights that drive LLM performance on single-cell data, and introduced scMPT, a model that combines scGPT with LLM representations. scMPT achieved stronger, more consistent performance than its component models, which showed large performance gaps across datasets.
Single-cell foundation models such as scGPT represent a significant advancement in single-cell omics, with an ability to achieve state-of-the-art performance on various downstream biological tasks. However, these models are inherently limited in that a vast amount of information in biology exists as text, which they are unable to leverage. There have therefore been several recent works that propose the use of LLMs as an alternative to single-cell foundation models, achieving competitive results. However, there is little understanding of what factors drive this performance, along with a strong focus on using LLMs as an alternative, rather than complementary approach to single-cell foundation models. In this study, we therefore investigate what biological insights contribute toward the performance of LLMs when applied to single-cell data, and introduce scMPT; a model which leverages synergies between scGPT, and single-cell representations from LLMs that capture these insights. scMPT demonstrates stronger, more consistent performance than either of its component models, which frequently have large performance gaps between each other across datasets. We also experiment with alternate fusion methods, demonstrating the potential of combining specialized reasoning models with scGPT to improve performance. This study ultimately showcases the potential for LLMs to complement single-cell foundation models and drive improvements in single-cell analysis.