MURMUR: Modular Multi-Step Reasoning for Semi-Structured Data-to-Text Generation
This addresses the challenge of generating accurate and consistent text from semi-structured data for applications in data-to-text generation, representing an incremental advance by combining neural and symbolic methods.
The paper tackled the problem of low semantic coverage, hallucination, and logical inconsistency in text generation from semi-structured data like graphs or tables, proposing MURMUR, a neuro-symbolic modular approach that achieved significant improvements over few-shot baselines and generated 26% more logically consistent summaries on LogicNLG compared to direct prompting.
Prompting large language models has enabled significant recent progress in multi-step reasoning over text. However, when applied to text generation from semi-structured data (e.g., graphs or tables), these methods typically suffer from low semantic coverage, hallucination, and logical inconsistency. We propose MURMUR, a neuro-symbolic modular approach to text generation from semi-structured data with multi-step reasoning. MURMUR is a best-first search method that generates reasoning paths using: (1) neural and symbolic modules with specific linguistic and logical skills, (2) a grammar whose production rules define valid compositions of modules, and (3) value functions that assess the quality of each reasoning step. We conduct experiments on two diverse data-to-text generation tasks like WebNLG and LogicNLG. These tasks differ in their data representations (graphs and tables) and span multiple linguistic and logical skills. MURMUR obtains significant improvements over recent few-shot baselines like direct prompting and chain-of-thought prompting, while also achieving comparable performance to fine-tuned GPT-2 on out-of-domain data. Moreover, human evaluation shows that MURMUR generates highly faithful and correct reasoning paths that lead to 26% more logically consistent summaries on LogicNLG, compared to direct prompting.