Outline of an Independent Systematic Blackbox Test for ML-based Systems
This addresses the need for reliable testing methods in ML applications, but appears incremental as it builds on existing test methods.
The paper tackles the problem of independently verifying the quality of ML models and systems, proposing a test procedure that accounts for their black-box and stochastic nature, and presents initial experimental results.
This article proposes a test procedure that can be used to test ML models and ML-based systems independently of the actual training process. In this way, the typical quality statements such as accuracy and precision of these models and system can be verified independently, taking into account their black box character and the immanent stochastic properties of ML models and their training data. The article presents first results from a set of test experiments and suggest extensions to existing test methods reflecting the stochastic nature of ML models and ML-based systems.