Structural shifts in institutional participation and collaboration within the AI arXiv preprint research ecosystem
It addresses the problem of understanding institutional divides in AI research collaboration for policymakers and researchers, though it is incremental in applying existing methods to new data.
This paper analyzed structural changes in the AI research landscape using arXiv preprints from 2021-2025, finding an unprecedented surge in publication output after ChatGPT's introduction, but academic-industry collaboration remained suppressed with a Normalized Collaboration Index significantly below baseline.
The emergence of large language models (LLMs) represents a significant technological shift within the scientific ecosystem, particularly within the field of artificial intelligence (AI). This paper examines structural changes in the AI research landscape using a dataset of arXiv preprints (cs.AI) from 2021 through 2025. Given the rapid pace of AI development, the preprint ecosystem has become a critical barometer for real-time scientific shifts, often preceding formal peer-reviewed publication by months or years. By employing a multi-stage data collection and enrichment pipeline in conjunction with LLM-based institution classification, we analyze the evolution of publication volumes, author team sizes, and academic--industry collaboration patterns. Our results reveal an unprecedented surge in publication output following the introduction of ChatGPT, with academic institutions continuing to provide the largest volume of research. However, we observe that academic--industry collaboration is still suppressed, as measured by a Normalized Collaboration Index (NCI) that remains significantly below the random-mixing baseline across all major subfields. These findings highlight a continuing institutional divide and suggest that the capital-intensive nature of generative AI research may be reshaping the boundaries of scientific collaboration.