NABench: Large-Scale Benchmarks of Nucleotide Foundation Models for Fitness Prediction
This provides a standardized benchmark for researchers in nucleic acid modeling, supporting applications in RNA/DNA design and synthetic biology, but it is incremental as it builds on existing foundation models.
The authors tackled the problem of inconsistent benchmarking for nucleotide foundation models in fitness prediction by introducing NABench, a large-scale benchmark with 162 assays and 2.6 million sequences, which revealed performance heterogeneity across 29 models in various settings.
Nucleotide sequence variation can induce significant shifts in functional fitness. Recent nucleotide foundation models promise to predict such fitness effects directly from sequence, yet heterogeneous datasets and inconsistent preprocessing make it difficult to compare methods fairly across DNA and RNA families. Here we introduce NABench, a large-scale, systematic benchmark for nucleic acid fitness prediction. NABench aggregates 162 high-throughput assays and curates 2.6 million mutated sequences spanning diverse DNA and RNA families, with standardized splits and rich metadata. We show that NABench surpasses prior nucleotide fitness benchmarks in scale, diversity, and data quality. Under a unified evaluation suite, we rigorously assess 29 representative foundation models across zero-shot, few-shot prediction, transfer learning, and supervised settings. The results quantify performance heterogeneity across tasks and nucleic-acid types, demonstrating clear strengths and failure modes for different modeling choices and establishing strong, reproducible baselines. We release NABench to advance nucleic acid modeling, supporting downstream applications in RNA/DNA design, synthetic biology, and biochemistry. Our code is available at https://github.com/mrzzmrzz/NABench.