Lightweight Convolutional Representations for On-Device Natural Language Processing
This work addresses the challenge of on-device natural language processing for mobile and wearable devices, presenting an incremental improvement by focusing on input representation compression.
The paper tackled the problem of deploying deep neural networks on low-resource devices by proposing a lightweight convolutional representation that can be compressed up to 32x with minimal performance loss, showing gains over recurrent representations in resource metrics like model size and latency on a Samsung Galaxy S9.
The increasing computational and memory complexities of deep neural networks have made it difficult to deploy them on low-resource electronic devices (e.g., mobile phones, tablets, wearables). Practitioners have developed numerous model compression methods to address these concerns, but few have condensed input representations themselves. In this work, we propose a fast, accurate, and lightweight convolutional representation that can be swapped into any neural model and compressed significantly (up to 32x) with a negligible reduction in performance. In addition, we show gains over recurrent representations when considering resource-centric metrics (e.g., model file size, latency, memory usage) on a Samsung Galaxy S9.