How Far are We from Effective Context Modeling? An Exploratory Study on Semantic Parsing in Context
This work addresses the challenge of complex contextual phenomena in semantic parsing, providing insights for future research, though it is incremental as it adapts existing methods.
The study evaluated 13 context modeling methods for semantic parsing in context on two large datasets, achieving state-of-the-art performance with significant improvements.
Recently semantic parsing in context has received considerable attention, which is challenging since there are complex contextual phenomena. Previous works verified their proposed methods in limited scenarios, which motivates us to conduct an exploratory study on context modeling methods under real-world semantic parsing in context. We present a grammar-based decoding semantic parser and adapt typical context modeling methods on top of it. We evaluate 13 context modeling methods on two large complex cross-domain datasets, and our best model achieves state-of-the-art performances on both datasets with significant improvements. Furthermore, we summarize the most frequent contextual phenomena, with a fine-grained analysis on representative models, which may shed light on potential research directions. Our code is available at https://github.com/microsoft/ContextualSP.