PDFA Distillation via String Probability Queries
This work addresses the need for explainable AI by providing a method to extract interpretable PDFA models from neural networks, though it appears incremental as it builds on existing L# algorithms.
The authors tackled the problem of distilling probabilistic deterministic finite automata (PDFA) from neural networks to enhance explainability in machine learning, presenting an algorithm derived from L# that uses string probability queries and demonstrating its effectiveness on a public dataset with trained neural networks.
Probabilistic deterministic finite automata (PDFA) are discrete event systems modeling conditional probabilities over languages: Given an already seen sequence of tokens they return the probability of tokens of interest to appear next. These types of models have gained interest in the domain of explainable machine learning, where they are used as surrogate models for neural networks trained as language models. In this work we present an algorithm to distill PDFA from neural networks. Our algorithm is a derivative of the L# algorithm and capable of learning PDFA from a new type of query, in which the algorithm infers conditional probabilities from the probability of the queried string to occur. We show its effectiveness on a recent public dataset by distilling PDFA from a set of trained neural networks.