Regionally Additive Models: Explainable-by-design models minimizing feature interactions
This addresses the need for more accurate explainable-by-design models in machine learning applications where feature interactions are important, though it is incremental as it builds on GAMs.
The paper tackles the problem of Generalized Additive Models (GAMs) failing to capture feature interactions, leading to subpar accuracy, by proposing Regionally Additive Models (RAMs) that identify subregions with minimized interactions and fit components per subregion, resulting in improved expressiveness while retaining interpretability.
Generalized Additive Models (GAMs) are widely used explainable-by-design models in various applications. GAMs assume that the output can be represented as a sum of univariate functions, referred to as components. However, this assumption fails in ML problems where the output depends on multiple features simultaneously. In these cases, GAMs fail to capture the interaction terms of the underlying function, leading to subpar accuracy. To (partially) address this issue, we propose Regionally Additive Models (RAMs), a novel class of explainable-by-design models. RAMs identify subregions within the feature space where interactions are minimized. Within these regions, it is more accurate to express the output as a sum of univariate functions (components). Consequently, RAMs fit one component per subregion of each feature instead of one component per feature. This approach yields a more expressive model compared to GAMs while retaining interpretability. The RAM framework consists of three steps. Firstly, we train a black-box model. Secondly, using Regional Effect Plots, we identify subregions where the black-box model exhibits near-local additivity. Lastly, we fit a GAM component for each identified subregion. We validate the effectiveness of RAMs through experiments on both synthetic and real-world datasets. The results confirm that RAMs offer improved expressiveness compared to GAMs while maintaining interpretability.