Probabilistic Topic and Syntax Modeling with Part-of-Speech LDA
This addresses the challenge of integrating syntax and semantics in NLP, offering incremental improvements for tasks like unsupervised part-of-speech tagging.
The authors tackled the problem of jointly modeling semantic topics and syntactic patterns in text by introducing Part-of-Speech LDA (POSLDA), which resulted in improved unsupervised part-of-speech tagging performance beyond the state of the art.
This article presents a probabilistic generative model for text based on semantic topics and syntactic classes called Part-of-Speech LDA (POSLDA). POSLDA simultaneously uncovers short-range syntactic patterns (syntax) and long-range semantic patterns (topics) that exist in document collections. This results in word distributions that are specific to both topics (sports, education, ...) and parts-of-speech (nouns, verbs, ...). For example, multinomial distributions over words are uncovered that can be understood as "nouns about weather" or "verbs about law". We describe the model and an approximate inference algorithm and then demonstrate the quality of the learned topics both qualitatively and quantitatively. Then, we discuss an NLP application where the output of POSLDA can lead to strong improvements in quality: unsupervised part-of-speech tagging. We describe algorithms for this task that make use of POSLDA-learned distributions that result in improved performance beyond the state of the art.