TIGS: An Inference Algorithm for Text Infilling with Gradient Search
This addresses a bottleneck in text infilling for natural language generation applications, offering a broadly applicable solution, though it is incremental as it builds on existing generative models.
The paper tackles the challenge of generating missing text in sentences or paragraphs using existing neural sequence models, proposing a gradient search-based inference algorithm that consistently outperforms strong baselines across various text infilling tasks with different mask ratios and strategies.
Text infilling is defined as a task for filling in the missing part of a sentence or paragraph, which is suitable for many real-world natural language generation scenarios. However, given a well-trained sequential generative model, generating missing symbols conditioned on the context is challenging for existing greedy approximate inference algorithms. In this paper, we propose an iterative inference algorithm based on gradient search, which is the first inference algorithm that can be broadly applied to any neural sequence generative models for text infilling tasks. We compare the proposed method with strong baselines on three text infilling tasks with various mask ratios and different mask strategies. The results show that our proposed method is effective and efficient for fill-in-the-blank tasks, consistently outperforming all baselines.