AI Pangaea: Unifying Intelligence Islands for Adapting Myriad Tasks
This work addresses the challenge of artificial general intelligence by unifying disparate AI models, though it appears incremental as it builds on existing pre-training and multi-task learning approaches.
The authors tackled the problem of isolated AI models limited to specific tasks by proposing Pangaea, a unified model that encodes any data into a unified format and pre-trains on 296 datasets across diverse modalities, achieving remarkable generalization across 45 general tasks and 15 scientific tasks.
The pursuit of artificial general intelligence continuously demands generalization in one model across myriad tasks, even those not seen before. However, current AI models are isolated from each other for being limited to specific tasks, now first defined as Intelligence Islands. To unify Intelligence Islands into one, we propose Pangaea, the first AI supercontinent akin to the geological Pangaea. Pangaea encodes any data into a unified format and accumulates universal knowledge through pre-training on 296 datasets across diverse modalities. Eventually, it demonstrates remarkable generalization across 45 general tasks and 15 scientific tasks encompassing a wide range of scientific subjects. By investigating Pangaea deeper, the scaling effect of modality is revealed, quantifying the universal knowledge accumulation across modalities as the cumulative distribution function of a geometric distribution. On the whole, Pangaea shows strong potential to handle myriad tasks, indicating a new direction toward artificial general intelligence.