Incorporating Consistency Verification into Neural Data-to-Document Generation
This addresses data consistency issues in document generation for applications like automated reporting, though it is incremental.
The paper tackles the problem of neural data-to-document generation models producing texts that often don't match input data, proposing a training framework that uses consistency verification via relation extraction and reinforcement learning. Experimental results on the ROTOWIRE dataset show improvements in various metrics.
Recent neural models for data-to-document generation have achieved remarkable progress in producing fluent and informative texts. However, large proportions of generated texts do not actually conform to the input data. To address this issue, we propose a new training framework which attempts to verify the consistency between the generated texts and the input data to guide the training process. To measure the consistency, a relation extraction model is applied to check information overlaps between the input data and the generated texts. The non-differentiable consistency signal is optimized via reinforcement learning. Experimental results on a recently released challenging dataset ROTOWIRE show improvements from our framework in various metrics.