The Software Heritage Graph Dataset: Large-scale Analysis of Public Software Development History
This dataset provides researchers with unprecedented access to the long tail of public software development, addressing a gap in data availability for large-scale software analysis.
The authors tackled the problem of limited access to large-scale public software development history by creating the Software Heritage Graph Dataset, which includes over five billion source code files and one billion commits from more than 80 million projects, enabling analysis beyond just popular repositories.
Software Heritage is the largest existing public archive of software source code and accompanying development history. It spans more than five billion unique source code files and one billion unique commits , coming from more than 80 million software projects. These software artifacts were retrieved from major collaborative development platforms (e.g., GitHub, GitLab) and package repositories (e.g., PyPI, Debian, NPM), and stored in a uniform representation linking together source code files, directories, commits, and full snapshots of version control systems (VCS) repositories as observed by Software Heritage during periodic crawls. This dataset is unique in terms of accessibility and scale, and allows to explore a number of research questions on the long tail of public software development, instead of solely focusing on ''most starred'' repositories as it often happens.