Beyond Additive Decompositions: Interpretability Through Separability
This work is significant for researchers and practitioners in interpretable machine learning, offering an alternative to additive models that can better capture complex interactions in data.
This paper addresses the limitations of additive representations in interpretable machine learning, which struggle with strong interactions due to signal cancellation and off-support extrapolation. The authors propose Tensor Separation Learning (TSL), a regression model that learns a sum of rank-1 products of univariate per-feature functions, demonstrating competitive performance with black-box models on regression benchmarks.
Interpretable machine learning requires models that are accurate and structurally faithful to the data.Existing explainability methods rely heavily on additive representations (e.g., Generalized Additive Models (GAMs), SHapley Additive exPlanations (SHAP), functional ANOVA), which can suffer from signal cancellation and off-support extrapolation in the presence of strong interactions. We propose Tensor Separation Learning (TSL), a regression model that learns a sum of rank-1 products of univariate per-feature functions via a stagewise greedy procedure with orthogonal refitting. By enforcing separability, TSL avoids the information loss inherent in additive projections caused by marginalizing higher-order interactions. The learned TSL model can be fully reconstructed from first-order partial dependence functions, up to constant factors. This stage-wise correspondence ensures that the resulting visualizations are faithful to the fitted components. We establish approximation-rate guarantees for functions with bounded mixed $p$-th order partial derivatives and demonstrate that TSL competes with black-box models on regression benchmarks.