Attention-like feature explanation for tabular data
This provides a method for improving interpretability in machine learning for tabular data, which is incremental as it builds on existing explanation techniques.
The authors tackled the problem of explaining black-box model predictions for tabular data by proposing AFEX, a system that uses neural subnetworks and an attention-like mechanism to generate local and global feature explanations, achieving results that illustrate its effectiveness on synthetic and real data.
A new method for local and global explanation of the machine learning black-box model predictions by tabular data is proposed. It is implemented as a system called AFEX (Attention-like Feature EXplanation) and consisting of two main parts. The first part is a set of the one-feature neural subnetworks which aim to get a specific representation for every feature in the form of a basis of shape functions. The subnetworks use shortcut connections with trainable parameters to improve the network performance. The second part of AFEX produces shape functions of features as the weighted sum of the basis shape functions where weights are computed by using an attention-like mechanism. AFEX identifies pairwise interactions between features based on pairwise multiplications of shape functions corresponding to different features. A modification of AFEX with incorporating an additional surrogate model which approximates the black-box model is proposed. AFEX is trained end-to-end on a whole dataset only once such that it does not require to train neural networks again in the explanation stage. Numerical experiments with synthetic and real data illustrate AFEX.