Language-Enhanced Representation Learning for Single-Cell Transcriptomics
This addresses a bottleneck in single-cell RNA sequencing analysis for biomedical researchers by integrating textual knowledge, though it appears incremental as an enhancement to existing scLLM approaches.
The paper tackles the limitation of single-cell large language models that ignore textual biological knowledge by proposing scMMGPT, a multimodal framework for language-enhanced representation learning in single-cell transcriptomics. The results show scMMGPT significantly outperforms baselines in cell annotation and clustering tasks and generalizes better in out-of-distribution scenarios.
Single-cell RNA sequencing (scRNA-seq) offers detailed insights into cellular heterogeneity. Recent advancements leverage single-cell large language models (scLLMs) for effective representation learning. These models focus exclusively on transcriptomic data, neglecting complementary biological knowledge from textual descriptions. To overcome this limitation, we propose scMMGPT, a novel multimodal framework designed for language-enhanced representation learning in single-cell transcriptomics. Unlike existing methods, scMMGPT employs robust cell representation extraction, preserving quantitative gene expression data, and introduces an innovative two-stage pre-training strategy combining discriminative precision with generative flexibility. Extensive experiments demonstrate that scMMGPT significantly outperforms unimodal and multimodal baselines across key downstream tasks, including cell annotation and clustering, and exhibits superior generalization in out-of-distribution scenarios.