Machine Learning in the Wild: Early Evidence of Non-Compliant ML-Automation in Open-Source Software
This addresses the problem of regulatory compliance in ML deployment for software developers, but it is incremental as it provides early evidence and groundwork rather than a solution.
The paper investigates the use of machine learning models in 173 open-source projects on GitHub across 16 domains, finding evidence of non-compliant usage that may violate terms of use and regulations, and aims to establish guidelines for developers.
The increasing availability of Machine Learning (ML) models, particularly foundation models, enables their use across a range of downstream applications, from scenarios with missing data to safety-critical contexts. This, in principle, may contravene not only the models' terms of use, but also governmental principles and regulations. This paper presents a preliminary investigation into the use of ML models by 173 open-source projects on GitHub, spanning 16 application domains. We evaluate whether models are used to make decisions, the scope of these decisions, and whether any post-processing measures are taken to reduce the risks inherent in fully autonomous systems. Lastly, we investigate the models' compliance with established terms of use. This study lays the groundwork for defining guidelines for developers and creating analysis tools that automatically identify potential regulatory violations in the use of ML models in software systems.