Weakly Supervised Semantic Parsing with Execution-based Spurious Program Filtering
This addresses a longstanding challenge in semantic parsing for NLP researchers, offering an incremental improvement by enhancing existing frameworks with a novel filtering approach.
The paper tackles the problem of spurious programs in weakly supervised semantic parsing by proposing a domain-agnostic filtering mechanism based on program execution results, which when applied to existing parsers, leads to significantly improved performances on benchmarks like Natural Language Visual Reasoning and WikiTableQuestions.
The problem of spurious programs is a longstanding challenge when training a semantic parser from weak supervision. To eliminate such programs that have wrong semantics but correct denotation, existing methods focus on exploiting similarities between examples based on domain-specific knowledge. In this paper, we propose a domain-agnostic filtering mechanism based on program execution results. Specifically, for each program obtained through the search process, we first construct a representation that captures the program's semantics as execution results under various inputs. Then, we run a majority vote on these representations to identify and filter out programs with significantly different semantics from the other programs. In particular, our method is orthogonal to the program search process so that it can easily augment any of the existing weakly supervised semantic parsing frameworks. Empirical evaluations on the Natural Language Visual Reasoning and WikiTableQuestions demonstrate that applying our method to the existing semantic parsers induces significantly improved performances.