The Importance of Category Labels in Grammar Induction with Child-directed Utterances
This work addresses the evaluation gap in grammar induction for computational linguistics, but it is incremental as it builds on existing methods by emphasizing labeled metrics.
The paper tackled the problem of evaluating grammar induction systems by showing that using labeled evaluation metrics reveals linguistically motivated predictions about grammar sparsity and categories, which are ignored in unlabeled evaluations. It demonstrated that depth-bounding remains effective with labeled evaluation on multilingual child-directed utterances.
Recent progress in grammar induction has shown that grammar induction is possible without explicit assumptions of language-specific knowledge. However, evaluation of induced grammars usually has ignored phrasal labels, an essential part of a grammar. Experiments in this work using a labeled evaluation metric, RH, show that linguistically motivated predictions about grammar sparsity and use of categories can only be revealed through labeled evaluation. Furthermore, depth-bounding as an implementation of human memory constraints in grammar inducers is still effective with labeled evaluation on multilingual transcribed child-directed utterances.