Modulating Language Models with Emotions
This work addresses the challenge of building empathetic NLP systems by enabling emotion modulation in language models, representing an incremental improvement over existing methods.
The paper tackled the problem of generating emotionally diverse and context-aware language by proposing modulated layer normalization, which outperformed prior baselines on the MojiTalk dataset in automatic and human evaluations while maintaining diversity, fluency, and coherence, and achieved competitive performance with only 10% of training data.
Generating context-aware language that embodies diverse emotions is an important step towards building empathetic NLP systems. In this paper, we propose a formulation of modulated layer normalization -- a technique inspired by computer vision -- that allows us to use large-scale language models for emotional response generation. In automatic and human evaluation on the MojiTalk dataset, our proposed modulated layer normalization method outperforms prior baseline methods while maintaining diversity, fluency, and coherence. Our method also obtains competitive performance even when using only 10% of the available training data.