Sparse Interaction Additive Networks via Feature Interaction Detection and Sparse Selection
This addresses the problem of balancing interpretability and performance in machine learning for tabular data, representing an incremental improvement by bridging simple models and neural networks.
The paper tackles the performance gap between interpretable models like linear regression and powerful deep neural networks by developing Sparse Interaction Additive Networks (SIAN), which uses feature interaction detection and sparse selection to efficiently identify necessary feature combinations, achieving competitive performance on large-scale tabular datasets.
There is currently a large gap in performance between the statistically rigorous methods like linear regression or additive splines and the powerful deep methods using neural networks. Previous works attempting to close this gap have failed to fully investigate the exponentially growing number of feature combinations which deep networks consider automatically during training. In this work, we develop a tractable selection algorithm to efficiently identify the necessary feature combinations by leveraging techniques in feature interaction detection. Our proposed Sparse Interaction Additive Networks (SIAN) construct a bridge from these simple and interpretable models to fully connected neural networks. SIAN achieves competitive performance against state-of-the-art methods across multiple large-scale tabular datasets and consistently finds an optimal tradeoff between the modeling capacity of neural networks and the generalizability of simpler methods.