Compound Probabilistic Context-Free Grammars for Grammar Induction
This addresses grammar induction for unsupervised parsing, offering an incremental improvement over existing methods.
The paper tackles grammar induction by modeling sentences with compound probabilistic context-free grammars, where rule probabilities are modulated by per-sentence latent variables, and reports effectiveness in unsupervised parsing experiments on English and Chinese compared to state-of-the-art methods.
We study a formalization of the grammar induction problem that models sentences as being generated by a compound probabilistic context-free grammar. In contrast to traditional formulations which learn a single stochastic grammar, our grammar's rule probabilities are modulated by a per-sentence continuous latent variable, which induces marginal dependencies beyond the traditional context-free assumptions. Inference in this grammar is performed by collapsed variational inference, in which an amortized variational posterior is placed on the continuous variable, and the latent trees are marginalized out with dynamic programming. Experiments on English and Chinese show the effectiveness of our approach compared to recent state-of-the-art methods when evaluated on unsupervised parsing.