Leveraging Sentence Similarity in Natural Language Generation: Improving Beam Search using Range Voting
This addresses the issue of short and uninformative outputs in natural language generation for tasks like image captioning and machine translation, though it is an incremental improvement on existing beam search methods.
The paper tackles the problem of language generation models preferring short outputs by proposing a method that selects the most representative output using range voting and similarity measures, resulting in longer, more diverse sentences with higher BLEU scores and better human ratings.
We propose a method for natural language generation, choosing the most representative output rather than the most likely output. By viewing the language generation process from the voting theory perspective, we define representativeness using range voting and a similarity measure. The proposed method can be applied when generating from any probabilistic language model, including n-gram models and neural network models. We evaluate different similarity measures on an image captioning task and a machine translation task, and show that our method generates longer and more diverse sentences, providing a solution to the common problem of short outputs being preferred over longer and more informative ones. The generated sentences obtain higher BLEU scores, particularly when the beam size is large. We also perform a human evaluation on both tasks and find that the outputs generated using our method are rated higher.