Theme-aware generation model for chinese lyrics
This work addresses theme coherence in Chinese lyric generation, offering potential applications in chatbots and long paragraph generation, but it appears incremental as it builds on existing seq2seq and attention methods.
The authors tackled the problem of generating Chinese music lyrics with better theme connectivity and coherence by proposing a theme-aware language generation model. Their multi-channel seq2seq model with attention mechanism produced grammatically correct and semantically coherent lyrics aligned with selected themes.
With rapid development of neural networks, deep-learning has been extended to various natural language generation fields, such as machine translation, dialogue generation and even literature creation. In this paper, we propose a theme-aware language generation model for Chinese music lyrics, which improves the theme-connectivity and coherence of generated paragraphs greatly. A multi-channel sequence-to-sequence (seq2seq) model encodes themes and previous sentences as global and local contextual information. Moreover, attention mechanism is incorporated for sequence decoding, enabling to fuse context into predicted next texts. To prepare appropriate train corpus, LDA (Latent Dirichlet Allocation) is applied for theme extraction. Generated lyrics is grammatically correct and semantically coherent with selected themes, which offers a valuable modelling method in other fields including multi-turn chatbots, long paragraph generation and etc.