ScDiVa: Masked Discrete Diffusion for Joint Modeling of Single-Cell Identity and Expression
This work addresses the challenge of modeling single-cell data for biologists and computational researchers, offering a biologically coherent alternative to existing methods, though it is incremental as it adapts diffusion techniques to this domain.
The authors tackled the problem of generating single-cell RNA-seq profiles, which are high-dimensional and sparse, by proposing scDiVa, a masked discrete diffusion model that avoids ordering bias and error accumulation from autoregressive methods. The model, pre-trained on 59 million cells, achieved strong transfer performance in benchmarks like batch integration and cell type annotation.
Single-cell RNA-seq profiles are high-dimensional, sparse, and unordered, causing autoregressive generation to impose an artificial ordering bias and suffer from error accumulation. To address this, we propose scDiVa, a masked discrete diffusion foundation model that aligns generation with the dropout-like corruption process by defining a continuous-time forward masking mechanism in token space. ScDiVa features a bidirectional denoiser that jointly models discrete gene identities and continuous values, utilizing entropy-normalized serialization and a latent anchor token to maximize information efficiency and preserve global cell identity. The model is trained via depth-invariant time sampling and a dual denoising objective to simulate varying sparsity levels while ensuring precise recovery of both identity and magnitude. Pre-trained on 59 million cells, scDiVa achieves strong transfer performance across major benchmarks, including batch integration, cell type annotation, and perturbation response prediction. These results suggest that masked discrete diffusion serves as a biologically coherent and effective alternative to autoregression.