Exploring Story Generation with Multi-task Objectives in Variational Autoencoders
This work addresses story generation for NLP applications, but it is incremental as it builds on existing models like GPT-2 and VAEs.
The paper tackled the problem of generating consistent and diverse stories by combining BERT and GPT-2 in a variational autoencoder with multi-task objectives, resulting in better quality-diversity trade-offs, less repetitive content, and a more informative latent variable.
GPT-2 has been frequently adapted in story generation models as it provides powerful generative capability. However, it still fails to generate consistent stories and lacks diversity. Current story generation models leverage additional information such as plots or commonsense into GPT-2 to guide the generation process. These approaches focus on improving generation quality of stories while our work look at both quality and diversity. We explore combining BERT and GPT-2 to build a variational autoencoder (VAE), and extend it by adding additional objectives to learn global features such as story topic and discourse relations. Our evaluations show our enhanced VAE can provide better quality and diversity trade off, generate less repetitive story content and learn a more informative latent variable.