Transcoder-based Circuit Analysis for Interpretable Single-Cell Foundation Models
This work addresses interpretability for researchers using single-cell models, but it is incremental as it applies an existing transcoder method to a new domain.
The paper tackled the problem of low interpretability in single-cell foundation models by using transcoders to extract internal decision-making circuits from the cell2sentence model, demonstrating that these circuits correspond to real-world biological mechanisms.
Single-cell foundation models (scFMs) have demonstrated state-of-the-art performance on various tasks, such as cell-type annotation and perturbation response prediction, by learning gene regulatory networks from large-scale transcriptome data. However, a significant challenge remains: the decision-making processes of these models are less interpretable compared to traditional methods like differential gene expression analysis. Recently, transcoders have emerged as a promising approach for extracting interpretable decision circuits from large language models (LLMs). In this work, we train a transcoder on the cell2sentence (C2S) model, a state-of-the-art scFM. By leveraging the trained transcoder, we extract internal decision-making circuits from the C2S model. We demonstrate that the discovered circuits correspond to real-world biological mechanisms, confirming the potential of transcoders to uncover biologically plausible pathways within complex single-cell models.