Policy Gradient as a Proxy for Dynamic Oracles in Constituency Parsing
This provides a parser-agnostic alternative for training constituency parsers, addressing a bottleneck in parsing research, though it is incremental as it builds on existing policy gradient and dynamic oracle concepts.
The paper tackled the problem of training constituency parsers without requiring custom dynamic oracles by using a policy gradient method, which directly optimizes for tree-level metrics like F1 and reduces exposure bias. The method outperformed static oracle training on four parsers in three languages, often recapturing much of the performance gain from dynamic oracles.
Dynamic oracles provide strong supervision for training constituency parsers with exploration, but must be custom defined for a given parser's transition system. We explore using a policy gradient method as a parser-agnostic alternative. In addition to directly optimizing for a tree-level metric such as F1, policy gradient has the potential to reduce exposure bias by allowing exploration during training; moreover, it does not require a dynamic oracle for supervision. On four constituency parsers in three languages, the method substantially outperforms static oracle likelihood training in almost all settings. For parsers where a dynamic oracle is available (including a novel oracle which we define for the transition system of Dyer et al. 2016), policy gradient typically recaptures a substantial fraction of the performance gain afforded by the dynamic oracle.