A Survey on Neural Network Language Models
It provides a comprehensive overview for researchers in NLP, but it is incremental as it synthesizes existing work without new experimental results.
This paper surveys neural network language models (NNLMs), which address the curse of dimensionality and enhance traditional language models, summarizing their structures, improvements, corpora, toolkits, and future research directions.
As the core component of Natural Language Processing (NLP) system, Language Model (LM) can provide word representation and probability indication of word sequences. Neural Network Language Models (NNLMs) overcome the curse of dimensionality and improve the performance of traditional LMs. A survey on NNLMs is performed in this paper. The structure of classic NNLMs is described firstly, and then some major improvements are introduced and analyzed. We summarize and compare corpora and toolkits of NNLMs. Further, some research directions of NNLMs are discussed.