Scalable Interpretability via Polynomials
This provides a scalable and high-performance interpretable alternative to DNNs for real-world tasks, addressing a key bottleneck in deploying interpretable models at scale.
The paper tackles the problem of interpretable machine learning lacking expressive power and scalability compared to uninterpretable methods like DNNs, by introducing Scalable Polynomial Additive Models (SPAM) that use tensor rank decompositions of polynomials to model all higher-order feature interactions without combinatorial explosion. The result is that SPAM outperforms current interpretable approaches and matches DNN/XGBoost performance on real-world benchmarks with up to hundreds of thousands of features, while being demonstrably more interpretable in human evaluations.
Generalized Additive Models (GAMs) have quickly become the leading choice for inherently-interpretable machine learning. However, unlike uninterpretable methods such as DNNs, they lack expressive power and easy scalability, and are hence not a feasible alternative for real-world tasks. We present a new class of GAMs that use tensor rank decompositions of polynomials to learn powerful, {\em inherently-interpretable} models. Our approach, titled Scalable Polynomial Additive Models (SPAM) is effortlessly scalable and models {\em all} higher-order feature interactions without a combinatorial parameter explosion. SPAM outperforms all current interpretable approaches, and matches DNN/XGBoost performance on a series of real-world benchmarks with up to hundreds of thousands of features. We demonstrate by human subject evaluations that SPAMs are demonstrably more interpretable in practice, and are hence an effortless replacement for DNNs for creating interpretable and high-performance systems suitable for large-scale machine learning. Source code is available at https://github.com/facebookresearch/nbm-spam.