Creating Powerful and Interpretable Models with Regression Networks
This addresses the need for interpretable models in machine learning, offering a solution for domains where understanding predictions is crucial, though it builds incrementally on existing hybrid methods.
The paper tackles the problem of creating interpretable yet powerful models by proposing Regression Networks, which combine neural networks with regression analysis and incorporate interactions, achieving state-of-the-art performance for interpretable models on benchmark datasets and matching dense neural network power.
As the discipline has evolved, research in machine learning has been focused more and more on creating more powerful neural networks, without regard for the interpretability of these networks. Such "black-box models" yield state-of-the-art results, but we cannot understand why they make a particular decision or prediction. Sometimes this is acceptable, but often it is not. We propose a novel architecture, Regression Networks, which combines the power of neural networks with the understandability of regression analysis. While some methods for combining these exist in the literature, our architecture generalizes these approaches by taking interactions into account, offering the power of a dense neural network without forsaking interpretability. We demonstrate that the models exceed the state-of-the-art performance of interpretable models on several benchmark datasets, matching the power of a dense neural network. Finally, we discuss how these techniques can be generalized to other neural architectures, such as convolutional and recurrent neural networks.