Content Modeling Using Latent Permutations
This work addresses discourse modeling for natural language processing applications, representing an incremental advancement in topic modeling methodology.
The authors tackled the problem of learning discourse-level document structure by developing a novel Bayesian topic model that constrains latent topic assignments using insights from discourse theory. Their method achieved substantial performance improvements over previous approaches on three complementary discourse-level tasks.
We present a novel Bayesian topic model for learning discourse-level document structure. Our model leverages insights from discourse theory to constrain latent topic assignments in a way that reflects the underlying organization of document topics. We propose a global model in which both topic selection and ordering are biased to be similar across a collection of related documents. We show that this space of orderings can be effectively represented using a distribution over permutations called the Generalized Mallows Model. We apply our method to three complementary discourse-level tasks: cross-document alignment, document segmentation, and information ordering. Our experiments show that incorporating our permutation-based model in these applications yields substantial improvements in performance over previously proposed methods.