A Formal Language Approach to Explaining RNNs
This addresses the interpretability challenge for RNNs in domains like verification and AI safety, though it is incremental as it builds on formal language and synthesis techniques.
The paper tackles the problem of explaining recurrent neural network (RNN) decisions by introducing LEXR, a framework that uses Linear Temporal Logic (LTL) to generate human-interpretable explanations, achieving more accurate and easier-to-understand results compared to existing automata-based methods.
This paper presents LEXR, a framework for explaining the decision making of recurrent neural networks (RNNs) using a formal description language called Linear Temporal Logic (LTL). LTL is the de facto standard for the specification of temporal properties in the context of formal verification and features many desirable properties that make the generated explanations easy for humans to interpret: it is a descriptive language, it has a variable-free syntax, and it can easily be translated into plain English. To generate explanations, LEXR follows the principle of counterexample-guided inductive synthesis and combines Valiant's probably approximately correct learning (PAC) with constraint solving. We prove that LEXR's explanations satisfy the PAC guarantee (provided the RNN can be described by LTL) and show empirically that these explanations are more accurate and easier-to-understand than the ones generated by recent algorithms that extract deterministic finite automata from RNNs.