Foundations of Symbolic Languages for Model Interpretability
This work addresses the problem of model interpretability for ML practitioners by providing a foundational language, though it is incremental as it builds on existing queries and focuses on specific model classes.
The paper tackles the need for flexible and reliable interpretability methods in machine learning by proposing FOIL, a logic-based declarative language for specifying explainability queries, and demonstrates its practical feasibility through a prototype implementation.
Several queries and scores have recently been proposed to explain individual predictions over ML models. Given the need for flexible, reliable, and easy-to-apply interpretability methods for ML models, we foresee the need for developing declarative languages to naturally specify different explainability queries. We do this in a principled way by rooting such a language in a logic, called FOIL, that allows for expressing many simple but important explainability queries, and might serve as a core for more expressive interpretability languages. We study the computational complexity of FOIL queries over two classes of ML models often deemed to be easily interpretable: decision trees and OBDDs. Since the number of possible inputs for an ML model is exponential in its dimension, the tractability of the FOIL evaluation problem is delicate but can be achieved by either restricting the structure of the models or the fragment of FOIL being evaluated. We also present a prototype implementation of FOIL wrapped in a high-level declarative language and perform experiments showing that such a language can be used in practice.