Narrative Text Generation with a Latent Discrete Plan
This work addresses story generation for NLP applications by introducing an unsupervised method to induce discrete plans, offering an incremental improvement over supervised approaches.
The authors tackled the problem of generating coherent stories by proposing a deep latent variable model that samples anchor words per sentence as a latent plan, trained unsupervisedly. Human evaluations showed their model outperformed baselines without plans and matched or exceeded those with external supervision, with favorable scores on perplexity, diversity, and plan control.
Past work on story generation has demonstrated the usefulness of conditioning on a generation plan to generate coherent stories. However, these approaches have used heuristics or off-the-shelf models to first tag training stories with the desired type of plan, and then train generation models in a supervised fashion. In this paper, we propose a deep latent variable model that first samples a sequence of anchor words, one per sentence in the story, as part of its generative process. During training, our model treats the sequence of anchor words as a latent variable and attempts to induce anchoring sequences that help guide generation in an unsupervised fashion. We conduct experiments with several types of sentence decoder distributions: left-to-right and non-monotonic, with different degrees of restriction. Further, since we use amortized variational inference to train our model, we introduce two corresponding types of inference network for predicting the posterior on anchor words. We conduct human evaluations which demonstrate that the stories produced by our model are rated better in comparison with baselines which do not consider story plans, and are similar or better in quality relative to baselines which use external supervision for plans. Additionally, the proposed model gets favorable scores when evaluated on perplexity, diversity, and control of story via discrete plan.