Incremental Permutation Feature Importance (iPFI): Towards Online Explanations on Data Streams
This work addresses the need for online explanations in data streams for applications requiring real-time interpretability, though it is incremental as it adapts an existing method to a new setting.
The paper tackled the problem of computing feature importance measures in dynamic, incremental learning scenarios, proposing an efficient model-agnostic algorithm called iPFI that approximates permutation feature importance with theoretical guarantees on expectation and variance, and validated it experimentally on benchmark data with and without concept drift.
Explainable Artificial Intelligence (XAI) has mainly focused on static learning scenarios so far. We are interested in dynamic scenarios where data is sampled progressively, and learning is done in an incremental rather than a batch mode. We seek efficient incremental algorithms for computing feature importance (FI) measures, specifically, an incremental FI measure based on feature marginalization of absent features similar to permutation feature importance (PFI). We propose an efficient, model-agnostic algorithm called iPFI to estimate this measure incrementally and under dynamic modeling conditions including concept drift. We prove theoretical guarantees on the approximation quality in terms of expectation and variance. To validate our theoretical findings and the efficacy of our approaches compared to traditional batch PFI, we conduct multiple experimental studies on benchmark data with and without concept drift.