Scalable and Robust Construction of Topical Hierarchies
This addresses the need for scalable and robust topic hierarchy construction in knowledge engineering, with incremental improvements in efficiency and usability.
The paper tackles the problem of automatically generating high-quality topical hierarchies from text collections, achieving a reduction in construction time by several orders of magnitude and enabling interactive revisions.
Automated generation of high-quality topical hierarchies for a text collection is a dream problem in knowledge engineering with many valuable applications. In this paper a scalable and robust algorithm is proposed for constructing a hierarchy of topics from a text collection. We divide and conquer the problem using a top-down recursive framework, based on a tensor orthogonal decomposition technique. We solve a critical challenge to perform scalable inference for our newly designed hierarchical topic model. Experiments with various real-world datasets illustrate its ability to generate robust, high-quality hierarchies efficiently. Our method reduces the time of construction by several orders of magnitude, and its robust feature renders it possible for users to interactively revise the hierarchy.