Can sparse autoencoders make sense of gene expression latent variable models?
This work addresses the challenge of interpretability in complex biological data for researchers in computational biology, though it is incremental as it adapts an existing method from language models to a new domain.
The paper tackles the problem of interpreting latent variables in gene expression models by applying sparse autoencoders (SAEs) to biological data, showing that SAEs can extract ground truth generative variables from simulated data and uncover subtle biological signals in single-cell models.
Sparse autoencoders (SAEs) have lately been used to uncover interpretable latent features in large language models. By projecting dense embeddings into a much higher-dimensional and sparse space, learned features become disentangled and easier to interpret. This work explores the potential of SAEs for decomposing embeddings in complex and high-dimensional biological data. Using simulated data, it outlines the efficacy, hyperparameter landscape, and limitations of SAEs when it comes to extracting ground truth generative variables from latent space. The application to embeddings from pretrained single-cell models shows that SAEs can find and steer key biological processes and even uncover subtle biological signals that might otherwise be missed. This work further introduces scFeatureLens, an automated interpretability approach for linking SAE features and biological concepts from gene sets to enable large-scale analysis and hypothesis generation in single-cell gene expression models.