Bidirectional Mamba for Single-Cell Data: Efficient Context Learning with Biological Fidelity
This work addresses the problem of inefficient and suboptimal handling of single-cell data for researchers in computational biology, offering a practical alternative to transformer-based methods, though it is incremental as it builds on existing state space modeling techniques.
The paper tackled the computational challenges of single-cell RNA sequencing data by introducing GeneMamba, a scalable foundation model based on state space modeling, which achieved strong performance in tasks like multi-batch integration and cell type annotation with linear-time complexity, pretrained on nearly 30 million cells.
Single-cell RNA sequencing (scRNA-seq) enables high-resolution analysis of cellular heterogeneity, but its complexity, which is marked by high dimensionality, sparsity, and batch effects, which poses major computational challenges. Transformer-based models have made significant advances in this domain but are often limited by their quadratic complexity and suboptimal handling of long-range dependencies. In this work, we introduce GeneMamba, a scalable and efficient foundation model for single-cell transcriptomics built on state space modeling. Leveraging the Bi-Mamba architecture, GeneMamba captures bidirectional gene context with linear-time complexity, offering substantial computational gains over transformer baselines. The model is pretrained on nearly 30 million cells and incorporates biologically informed objectives, including pathway-aware contrastive loss and rank-based gene encoding. We evaluate GeneMamba across diverse tasks, including multi-batch integration, cell type annotation, and gene-gene correlation, demonstrating strong performance, interpretability, and robustness. These results position GeneMamba as a practical and powerful alternative to transformer-based methods, advancing the development of biologically grounded, scalable tools for large-scale single-cell data analysis.