Representer Point Selection for Explaining Regularized High-dimensional Models
This work addresses the need for interpretability in complex models like sparse and low-rank regularized models, offering a method to explain predictions in terms of training data contributions, though it appears incremental as it builds on representer theorems.
The authors tackled the problem of explaining predictions from regularized high-dimensional models by introducing high-dimensional representers, which assign importance weights to training samples, and demonstrated empirical performance on real-world datasets including binary classification and recommender systems.
We introduce a novel class of sample-based explanations we term high-dimensional representers, that can be used to explain the predictions of a regularized high-dimensional model in terms of importance weights for each of the training samples. Our workhorse is a novel representer theorem for general regularized high-dimensional models, which decomposes the model prediction in terms of contributions from each of the training samples: with positive (negative) values corresponding to positive (negative) impact training samples to the model's prediction. We derive consequences for the canonical instances of $\ell_1$ regularized sparse models, and nuclear norm regularized low-rank models. As a case study, we further investigate the application of low-rank models in the context of collaborative filtering, where we instantiate high-dimensional representers for specific popular classes of models. Finally, we study the empirical performance of our proposed methods on three real-world binary classification datasets and two recommender system datasets. We also showcase the utility of high-dimensional representers in explaining model recommendations.