Improving Factual Accuracy of Neural Table-to-Text Output by Addressing Input Problems in ToTTo
This addresses the issue of factual inaccuracies in text generation for users relying on automated table-to-text systems, but it is incremental as it focuses on specific input problems in a dataset.
The study tackled the problem of factual errors in neural table-to-text models by tracing output hallucinations to input issues, and found that fixing these inputs reduced factual errors by 52% to 76% across models.
Neural Table-to-Text models tend to hallucinate, producing texts that contain factual errors. We investigate whether such errors in the output can be traced back to problems with the input. We manually annotated 1,837 texts generated by multiple models in the politics domain of the ToTTo dataset. We identify the input problems that are responsible for many output errors and show that fixing these inputs reduces factual errors by between 52% and 76% (depending on the model). In addition, we observe that models struggle in processing tabular inputs that are structured in a non-standard way, particularly when the input lacks distinct row and column values or when the column headers are not correctly mapped to corresponding values.