Efficient Transfer Learning Schemes for Personalized Language Modeling using Recurrent Neural Network
This work addresses privacy-preserving personalization for mobile devices, though it appears incremental as it builds on existing transfer learning and RNN-LSTM methods.
The paper tackles the problem of creating personalized language models with limited user data and computing resources, achieving results that more closely match individual language styles in both qualitative and quantitative evaluations.
In this paper, we propose an efficient transfer leaning methods for training a personalized language model using a recurrent neural network with long short-term memory architecture. With our proposed fast transfer learning schemes, a general language model is updated to a personalized language model with a small amount of user data and a limited computing resource. These methods are especially useful for a mobile device environment while the data is prevented from transferring out of the device for privacy purposes. Through experiments on dialogue data in a drama, it is verified that our transfer learning methods have successfully generated the personalized language model, whose output is more similar to the personal language style in both qualitative and quantitative aspects.