Optimizing for Measure of Performance in Max-Margin Parsing
This addresses the challenge of better aligning training objectives with evaluation metrics in natural language parsing, though it is an incremental improvement on existing structured prediction methods.
The paper tackles the problem of optimizing directly for F1-score in constituency parsing by integrating it into a max-margin structural SVM framework, resulting in improved parsing accuracy compared to standard methods.
Many statistical learning problems in the area of natural language processing including sequence tagging, sequence segmentation and syntactic parsing has been successfully approached by means of structured prediction methods. An appealing property of the corresponding discriminative learning algorithms is their ability to integrate the loss function of interest directly into the optimization process, which potentially can increase the resulting performance accuracy. Here, we demonstrate on the example of constituency parsing how to optimize for F1-score in the max-margin framework of structural SVM. In particular, the optimization is with respect to the original (not binarized) trees.