Semantic Parsing to Probabilistic Programs for Situated Question Answering
This addresses the problem of interpreting questions about environments like diagrams for AI systems, with incremental improvements in handling uncertainty and background knowledge.
The paper tackles situated question answering by introducing Parsing to Probabilistic Programs (P3), a model that uses semantic parses as probabilistic programs to handle environmental uncertainty, and it outperforms baselines on a dataset of 5000 science diagram questions.
Situated question answering is the problem of answering questions about an environment such as an image or diagram. This problem requires jointly interpreting a question and an environment using background knowledge to select the correct answer. We present Parsing to Probabilistic Programs (P3), a novel situated question answering model that can use background knowledge and global features of the question/environment interpretation while retaining efficient approximate inference. Our key insight is to treat semantic parses as probabilistic programs that execute nondeterministically and whose possible executions represent environmental uncertainty. We evaluate our approach on a new, publicly-released data set of 5000 science diagram questions, outperforming several competitive classical and neural baselines.