Analyzing the Evolution and Maintenance of ML Models on Hugging Face
It addresses the lack of comprehensive analysis on ML model maintenance and evolution on platforms like Hugging Face, offering insights for developers, but is incremental as it applies repository mining to a new dataset.
This study analyzed over 380,000 models on Hugging Face to explore community engagement, evolution, and maintenance, uncovering trends in domains, frameworks, and maintenance status through text analysis and commit classification.
Hugging Face (HF) has established itself as a crucial platform for the development and sharing of machine learning (ML) models. This repository mining study, which delves into more than 380,000 models using data gathered via the HF Hub API, aims to explore the community engagement, evolution, and maintenance around models hosted on HF, aspects that have yet to be comprehensively explored in the literature. We first examine the overall growth and popularity of HF, uncovering trends in ML domains, framework usage, authors grouping and the evolution of tags and datasets used. Through text analysis of model card descriptions, we also seek to identify prevalent themes and insights within the developer community. Our investigation further extends to the maintenance aspects of models, where we evaluate the maintenance status of ML models, classify commit messages into various categories (corrective, perfective, and adaptive), analyze the evolution across development stages of commits metrics and introduce a new classification system that estimates the maintenance status of models based on multiple attributes. This study aims to provide valuable insights about ML model maintenance and evolution that could inform future model development strategies on platforms like HF.