Inducing Regular Grammars Using Recurrent Neural Networks
This addresses grammar induction for computational linguistics, but it appears incremental as it applies existing neural methods to a known task with new insights.
The paper tackled the problem of learning a regular grammar from examples without prior assumptions, using a recurrent neural network to classify strings and extract a finite-state automaton, finding unexpected connections in the network's states that suggest generalization.
Grammar induction is the task of learning a grammar from a set of examples. Recently, neural networks have been shown to be powerful learning machines that can identify patterns in streams of data. In this work we investigate their effectiveness in inducing a regular grammar from data, without any assumptions about the grammar. We train a recurrent neural network to distinguish between strings that are in or outside a regular language, and utilize an algorithm for extracting the learned finite-state automaton. We apply this method to several regular languages and find unexpected results regarding the connections between the network's states that may be regarded as evidence for generalization.