Examination and Extension of Strategies for Improving Personalized Language Modeling via Interpolation
This work addresses enhancing personalized language models for users, but it is incremental as it builds on prior research in natural language interfaces.
The paper tackled improving personalized language modeling by developing strategies for model interpolation and handling out-of-vocabulary tokens, resulting in over 80% of users experiencing a perplexity lift with an average improvement of 5.2% per user.
In this paper, we detail novel strategies for interpolating personalized language models and methods to handle out-of-vocabulary (OOV) tokens to improve personalized language models. Using publicly available data from Reddit, we demonstrate improvements in offline metrics at the user level by interpolating a global LSTM-based authoring model with a user-personalized n-gram model. By optimizing this approach with a back-off to uniform OOV penalty and the interpolation coefficient, we observe that over 80% of users receive a lift in perplexity, with an average of 5.2% in perplexity lift per user. In doing this research we extend previous work in building NLIs and improve the robustness of metrics for downstream tasks.