A Transition System Abstraction Framework for Neural Network Dynamical System Models
This work addresses interpretability for complex dynamical systems like human behavior learning, but it appears incremental as it builds on existing abstraction and verification methods.
The paper tackles the problem of interpreting black-box neural network dynamical system models by proposing a transition system abstraction framework that converts them into interpretable transition systems, validated through human handwriting dynamics applications where specifications can be verified using Computational Tree Logic (CTL).
This paper proposes a transition system abstraction framework for neural network dynamical system models to enhance the model interpretability, with applications to complex dynamical systems such as human behavior learning and verification. To begin with, the localized working zone will be segmented into multiple localized partitions under the data-driven Maximum Entropy (ME) partitioning method. Then, the transition matrix will be obtained based on the set-valued reachability analysis of neural networks. Finally, applications to human handwriting dynamics learning and verification are given to validate our proposed abstraction framework, which demonstrates the advantages of enhancing the interpretability of the black-box model, i.e., our proposed framework is able to abstract a data-driven neural network model into a transition system, making the neural network model interpretable through verifying specifications described in Computational Tree Logic (CTL) languages.