Neural Networks Compression for Language Modeling
This work addresses the need for efficient language models in mobile settings, but it is incremental as it applies existing compression techniques without introducing new methods.
The paper tackles the problem of high space complexity and slow inference in LSTM-based language models, especially for mobile applications, by comparing pruning, quantization, low-rank factorization, and tensor train decomposition on the Penn Treebank dataset to reduce model size and improve inference speed.
In this paper, we consider several compression techniques for the language modeling problem based on recurrent neural networks (RNNs). It is known that conventional RNNs, e.g, LSTM-based networks in language modeling, are characterized with either high space complexity or substantial inference time. This problem is especially crucial for mobile applications, in which the constant interaction with the remote server is inappropriate. By using the Penn Treebank (PTB) dataset we compare pruning, quantization, low-rank factorization, tensor train decomposition for LSTM networks in terms of model size and suitability for fast inference.