Syntactic Structure from Deep Learning
It addresses the problem of understanding innate language structure for linguists and AI researchers, but is incremental as a survey.
The paper surveys studies on whether deep neural networks induce human-like grammatical knowledge from data, examining their syntactic abilities and implications for theoretical linguistics.
Modern deep neural networks achieve impressive performance in engineering applications that require extensive linguistic skills, such as machine translation. This success has sparked interest in probing whether these models are inducing human-like grammatical knowledge from the raw data they are exposed to, and, consequently, whether they can shed new light on long-standing debates concerning the innate structure necessary for language acquisition. In this article, we survey representative studies of the syntactic abilities of deep networks, and discuss the broader implications that this work has for theoretical linguistics.