Verifying Machine Learning Interpretability Requirements through Provenance
For ML engineers and requirements engineers, this work offers a practical approach to verify interpretability requirements, which was previously considered immeasurable.
The paper addresses the challenge of verifying interpretability as a non-functional requirement in ML models by using provenance data to create quantifiable functional requirements. It provides a method for ML engineers to save model and data provenance to make model behavior transparent, enabling verification of interpretability.
Machine Learning (ML) Engineering is a growing field that necessitates an increase in the rigor of ML development. It draws many ideas from software engineering and more specifically, from requirements engineering. Existing literature on ML Engineering defines quality models and Non-Functional Requirements (NFRs) specific to ML, in particular interpretability being one such NFR. However, a major challenge occurs in verifying ML NFRs, including interpretability. Although existing literature defines interpretability in terms of ML, it remains an immeasurable requirement, making it impossible to definitively confirm whether a model meets its interpretability requirement. This paper shows how ML provenance can be used to verify ML interpretability requirements. This work provides an approach for how ML engineers can save various types of model and data provenance to make the model's behavior transparent and interpretable. Saving this data forms the basis of quantifiable Functional Requirements (FRs) whose verification in turn verifies the interpretability NFR. Ultimately, this paper contributes a method to verify interpretability NFRs for ML models.