Logistic Regression, Neural Networks and Dempster-Shafer Theory: a New Perspective
This provides a new theoretical perspective for interpreting and enhancing existing classifiers, offering insights into feature roles and decision-making, though it is incremental as it builds on established methods.
The paper reinterprets logistic regression and neural networks through Dempster-Shafer theory, showing they can be viewed as converting features into mass functions and combining them with Dempster's rule, which allows distinguishing between lack of evidence and conflicting evidence to improve interpretability and enable alternative decision rules like interval dominance.
We revisit logistic regression and its nonlinear extensions, including multilayer feedforward neural networks, by showing that these classifiers can be viewed as converting input or higher-level features into Dempster-Shafer mass functions and aggregating them by Dempster's rule of combination. The probabilistic outputs of these classifiers are the normalized plausibilities corresponding to the underlying combined mass function. This mass function is more informative than the output probability distribution. In particular, it makes it possible to distinguish between lack of evidence (when none of the features provides discriminant information) from conflicting evidence (when different features support different classes). This expressivity of mass functions allows us to gain insight into the role played by each input feature in logistic regression, and to interpret hidden unit outputs in multilayer neural networks. It also makes it possible to use alternative decision rules, such as interval dominance, which select a set of classes when the available evidence does not unambiguously point to a single class, thus trading reduced error rate for higher imprecision.