Transforming Feature Space to Interpret Machine Learning Models
This work addresses model interpretation difficulties for domain experts in fields like remote sensing, though it is incremental as it builds on existing diagnostic methods.
The paper tackles the challenge of interpreting machine-learning models with dependent features in high-dimensional spaces, such as in remote sensing landcover classification, by proposing a feature space transformation approach that enhances existing diagnostic tools, demonstrated in a case study with 46 features.
Model-agnostic tools for interpreting machine-learning models struggle to summarize the joint effects of strongly dependent features in high-dimensional feature spaces, which play an important role in pattern recognition, for example in remote sensing of landcover. This contribution proposes a novel approach that interprets machine-learning models through the lens of feature space transformations. It can be used to enhance unconditional as well as conditional post-hoc diagnostic tools including partial dependence plots, accumulated local effects plots, or permutation feature importance assessments. While the approach can also be applied to nonlinear transformations, we focus on linear ones, including principal component analysis (PCA) and a partial orthogonalization technique. Structured PCA and diagnostics along paths offer opportunities for representing domain knowledge. The new approach is implemented in the R package `wiml`, which can be combined with existing explainable machine-learning packages. A case study on remote-sensing landcover classification with 46 features is used to demonstrate the potential of the proposed approach for model interpretation by domain experts.