Anchor-Free Correlated Topic Modeling: Identifiability and Algorithm
This work addresses the limitation of anchor-word assumptions in topic modeling for text analysis, offering a more practical and scalable solution for researchers and practitioners dealing with large datasets.
The paper tackles the problem of topic modeling without requiring anchor words, which are often unrealistic, by proposing an anchor-free framework based on second-order moments that guarantees topic identification under milder conditions. The approach demonstrates improved robustness and favorable performance on TDT2 and Reuters-21578 corpora, with metrics like coherence and clustering accuracy showing gains compared to prior methods.
In topic modeling, many algorithms that guarantee identifiability of the topics have been developed under the premise that there exist anchor words -- i.e., words that only appear (with positive probability) in one topic. Follow-up work has resorted to three or higher-order statistics of the data corpus to relax the anchor word assumption. Reliable estimates of higher-order statistics are hard to obtain, however, and the identification of topics under those models hinges on uncorrelatedness of the topics, which can be unrealistic. This paper revisits topic modeling based on second-order moments, and proposes an anchor-free topic mining framework. The proposed approach guarantees the identification of the topics under a much milder condition compared to the anchor-word assumption, thereby exhibiting much better robustness in practice. The associated algorithm only involves one eigen-decomposition and a few small linear programs. This makes it easy to implement and scale up to very large problem instances. Experiments using the TDT2 and Reuters-21578 corpus demonstrate that the proposed anchor-free approach exhibits very favorable performance (measured using coherence, similarity count, and clustering accuracy metrics) compared to the prior art.