yProv4ML: Effortless Provenance Tracking for Machine Learning Systems
This addresses the problem of opaque model development for ML researchers and practitioners, though it is incremental as it builds on existing provenance concepts.
The paper tackles the lack of transparency in machine learning development by proposing yProv4ML, a framework that captures provenance information in PROV-JSON format with minimal code modifications, enabling better tracking of hyperparameters and other details.
The rapid growth of interest in large language models (LLMs) reflects their potential for flexibility and generalization, and attracted the attention of a diverse range of researchers. However, the advent of these techniques has also brought to light the lack of transparency and rigor with which development is pursued. In particular, the inability to determine the number of epochs and other hyperparameters in advance presents challenges in identifying the best model. To address this challenge, machine learning frameworks such as MLFlow can automate the collection of this type of information. However, these tools capture data using proprietary formats and pose little attention to lineage. This paper proposes yProv4ML, a framework to capture provenance information generated during machine learning processes in PROV-JSON format, with minimal code modifications.