Learning Neural Templates for Text Generation
This work addresses interpretability and control issues in text generation for applications like natural language processing, though it is incremental as it builds on existing neural methods.
The paper tackled the problems of uninterpretability and lack of controllability in neural encoder-decoder models for text generation by proposing a system with a hidden semi-Markov model decoder that learns latent, discrete templates, resulting in improved interpretability and controllability while achieving strong performance nearing that of encoder-decoder models on real datasets.
While neural, encoder-decoder models have had significant empirical success in text generation, there remain several unaddressed problems with this style of generation. Encoder-decoder models are largely (a) uninterpretable, and (b) difficult to control in terms of their phrasing or content. This work proposes a neural generation system using a hidden semi-markov model (HSMM) decoder, which learns latent, discrete templates jointly with learning to generate. We show that this model learns useful templates, and that these templates make generation both more interpretable and controllable. Furthermore, we show that this approach scales to real data sets and achieves strong performance nearing that of encoder-decoder text generation models.