Formal Language Theory Meets Modern NLP
It provides a conceptual overview for researchers interested in the intersection of formal languages and neural networks, but it is incremental as it synthesizes existing ideas without new findings.
This paper explores the connection between formal language theory and modern NLP, focusing on how formal analysis applies to deep learning methods, but it does not present new research results or concrete numbers.
NLP is deeply intertwined with the formal study of language, both conceptually and historically. Arguably, this connection goes all the way back to Chomsky's Syntactic Structures in 1957. It also still holds true today, with a strand of recent works building formal analysis of modern neural networks methods in terms of formal languages. In this document, I aim to explain background about formal languages as they relate to this recent work. I will by necessity ignore large parts of the rich history of this field, instead focusing on concepts connecting to modern deep learning-based NLP.