Neural Search Space in Gboard Decoder
This work addresses the sparsity issue in keyboard decoding for mobile users, representing an incremental improvement by integrating neural LMs into an existing system.
The paper tackled the sparsity problem of N-gram language models in Gboard Decoder by replacing them with a Neural Network LM and dynamically constructing the search space during decoding, resulting in a reduction of Words Modified Ratio by 0.26% to 1.19% across locales with acceptable latency increases.
Gboard Decoder produces suggestions by looking for paths that best match input touch points on the context aware search space, which is backed by the language Finite State Transducers (FST). The language FST is currently an N-gram language model (LM). However, N-gram LMs, limited in context length, are known to have sparsity problem under device model size constraint. In this paper, we propose \textbf{Neural Search Space} which substitutes the N-gram LM with a Neural Network LM (NN-LM) and dynamically constructs the search space during decoding. Specifically, we integrate the long range context awareness of NN-LM into the search space by converting its outputs given context, into the language FST at runtime. This involves language FST structure redesign, pruning strategy tuning, and data structure optimizations. Online experiments demonstrate improved quality results, reducing Words Modified Ratio by [0.26\%, 1.19\%] on various locales with acceptable latency increases. This work opens new avenues for further improving keyboard decoding quality by enhancing neural LM more directly.