scTranslation: A Comprehensive Benchmark for Single-Cell Multi-Omics Modality Translation
For researchers in single-cell multi-omics, this benchmark provides a standardized evaluation framework to compare and improve modality translation methods.
The paper introduces scTranslation, a benchmark for single-cell multi-omics modality translation, which includes diverse datasets, state-of-the-art models, and comprehensive evaluation metrics. It systematically evaluates model performance under various scenarios, revealing significant factors affecting translation quality.
Simultaneous measurement of multiple omics modalities in single cells enables researchers to gain a more comprehensive understanding of cellular states and regulatory mechanisms. However, due to high experimental costs, significant noise, and incomplete modality coverage, a variety of computational methods for modality translation have emerged in recent years. Despite the development of translation models, there is still a lack of systematic benchmark evaluation in terms of datasets, evaluation metrics, and influencing factors. To address this, we present scTranslation, a comprehensive benchmark for single-cell multi-omics modality translation tasks. It includes diverse translation datasets, integrates state-of-the-art models, and provides a comprehensive evaluation metrics. In addition, we assess model performance under different scenarios, such as feature selection, feature quality, and few-shot settings. These factors significantly affect model performance but have rarely been systematically studied before. Leveraging this benchmark, we conduct a large-scale study of current methods, report many insightful findings that open up new possibilities for future development. The benchmark is open-sourced to facilitate future research. The code is anonymously released at https://github.com/Bunnybeibei/scTranslation.