Integrating Regular Expressions with Neural Networks via DFA
This work addresses the challenge of efficiently incorporating human-designed rules into neural models for applications like intent classification, offering an incremental improvement over existing hybrid methods.
The paper tackles the problem of maintaining numerous hand-crafted rules in industry applications by integrating regular expressions (REs) into neural networks via deterministic finite automata (MDFAs) to capture rule-based features. The method achieves the best performance on the ATIS intent classification task compared to neural networks and other hybrid approaches, particularly when training data is small.
Human-designed rules are widely used to build industry applications. However, it is infeasible to maintain thousands of such hand-crafted rules. So it is very important to integrate the rule knowledge into neural networks to build a hybrid model that achieves better performance. Specifically, the human-designed rules are formulated as Regular Expressions (REs), from which the equivalent Minimal Deterministic Finite Automatons (MDFAs) are constructed. We propose to use the MDFA as an intermediate model to capture the matched RE patterns as rule-based features for each input sentence and introduce these additional features into neural networks. We evaluate the proposed method on the ATIS intent classification task. The experiment results show that the proposed method achieves the best performance compared to neural networks and four other methods that combine REs and neural networks when the training dataset is relatively small.