xRAI: Explainable Representations through AI
This work addresses the problem of understanding neural network decision-making by making the target function explicit, which is an incremental contribution to the field of AI explainability.
This paper introduces xRAI, a method to extract symbolic mathematical representations from trained neural networks. It uses an 'interpretation network' that takes network weights and biases as input and outputs a numerical representation of the function, which can then be converted into a symbolic form.
We present xRAI an approach for extracting symbolic representations of the mathematical function a neural network was supposed to learn from the trained network. The approach is based on the idea of training a so-called interpretation network that receives the weights and biases of the trained network as input and outputs the numerical representation of the function the network was supposed to learn that can be directly translated into a symbolic representation. We show that interpretation nets for different classes of functions can be trained on synthetic data offline using Boolean functions and low-order polynomials as examples. We show that the training is rather efficient and the quality of the results are promising. Our work aims to provide a contribution to the problem of better understanding neural decision making by making the target function explicit