SAFE ML: Surrogate Assisted Feature Extraction for Model Learning
This addresses the trade-off between accuracy and interpretability in machine learning, particularly for domains requiring transparent models, though it is incremental in combining existing concepts.
The paper tackles the problem of training interpretable and accurate models without time-consuming feature engineering by using elastic black-box surrogate models to create simpler, interpretable glass-box models, achieving competitive performance on tabular datasets.
Complex black-box predictive models may have high accuracy, but opacity causes problems like lack of trust, lack of stability, sensitivity to concept drift. On the other hand, interpretable models require more work related to feature engineering, which is very time consuming. Can we train interpretable and accurate models, without timeless feature engineering? In this article, we show a method that uses elastic black-boxes as surrogate models to create a simpler, less opaque, yet still accurate and interpretable glass-box models. New models are created on newly engineered features extracted/learned with the help of a surrogate model. We show applications of this method for model level explanations and possible extensions for instance level explanations. We also present an example implementation in Python and benchmark this method on a number of tabular data sets.