Defining and Extracting generalizable interaction primitives from DNNs
This addresses the challenge of faithful summarization of DNN knowledge for interpretability, but it is incremental as it builds on prior work to improve generalization.
The paper tackles the problem of extracting generalizable interaction primitives from deep neural networks (DNNs) to improve explainable AI, and the result shows that the new method extracts interactions that better reflect common knowledge across different DNNs.
Faithfully summarizing the knowledge encoded by a deep neural network (DNN) into a few symbolic primitive patterns without losing much information represents a core challenge in explainable AI. To this end, Ren et al. (2024) have derived a series of theorems to prove that the inference score of a DNN can be explained as a small set of interactions between input variables. However, the lack of generalization power makes it still hard to consider such interactions as faithful primitive patterns encoded by the DNN. Therefore, given different DNNs trained for the same task, we develop a new method to extract interactions that are shared by these DNNs. Experiments show that the extracted interactions can better reflect common knowledge shared by different DNNs.