DNAGPT: A Generalized Pre-trained Tool for Versatile DNA Sequence Analysis Tasks
This work addresses the problem of versatile DNA analysis for bioinformatics researchers, offering a generalized tool that improves over task-specific models, though it appears incremental as it builds on existing GPT architectures with added tasks.
The authors tackled the challenge of adapting pre-trained language models to diverse DNA sequence analysis tasks by proposing DNAGPT, a generalized model trained on over 200 billion base pairs from mammals, which demonstrated superior performance in tasks like genomic signal recognition and mRNA abundance regression compared to existing specialized models.
Pre-trained large language models demonstrate potential in extracting information from DNA sequences, yet adapting to a variety of tasks and data modalities remains a challenge. To address this, we propose DNAGPT, a generalized DNA pre-training model trained on over 200 billion base pairs from all mammals. By enhancing the classic GPT model with a binary classification task (DNA sequence order), a numerical regression task (guanine-cytosine content prediction), and a comprehensive token language, DNAGPT can handle versatile DNA analysis tasks while processing both sequence and numerical data. Our evaluation of genomic signal and region recognition, mRNA abundance regression, and artificial genomes generation tasks demonstrates DNAGPT's superior performance compared to existing models designed for specific downstream tasks, benefiting from pre-training using the newly designed model structure.