CLOct 12, 2016
Language Models with Pre-Trained (GloVe) Word Embeddings
arXiv:1610.03759v211 citations
Originality Synthesis-oriented
AI Analysis
This work is incremental, applying a known method (GRU with GloVe) to language modeling without introducing new techniques.
The authors trained a language model using a GRU-based RNN with pre-trained GloVe word embeddings, achieving results that were not explicitly quantified in the abstract.
In this work we implement a training of a Language Model (LM), using Recurrent Neural Network (RNN) and GloVe word embeddings, introduced by Pennigton et al. in [1]. The implementation is following the general idea of training RNNs for LM tasks presented in [2], but is rather using Gated Recurrent Unit (GRU) [3] for a memory cell, and not the more commonly used LSTM [4].