Peacock: Learning Long-Tail Topic Features for Industrial Applications
This work addresses the scalability limitations of topic modeling for large-scale industrial applications like search and advertising, though it is incremental as it builds on existing LDA methods.
The authors tackled the problem of scaling latent Dirichlet allocation (LDA) for industrial use by developing Peacock, a distributed system that learns big LDA models with at least 10^5 topics from 10^9 search queries, achieving significant improvements in search engine and online advertising systems serving hundreds of millions of users.
Latent Dirichlet allocation (LDA) is a popular topic modeling technique in academia but less so in industry, especially in large-scale applications involving search engine and online advertising systems. A main underlying reason is that the topic models used have been too small in scale to be useful; for example, some of the largest LDA models reported in literature have up to $10^3$ topics, which cover difficultly the long-tail semantic word sets. In this paper, we show that the number of topics is a key factor that can significantly boost the utility of topic-modeling systems. In particular, we show that a "big" LDA model with at least $10^5$ topics inferred from $10^9$ search queries can achieve a significant improvement on industrial search engine and online advertising systems, both of which serving hundreds of millions of users. We develop a novel distributed system called Peacock to learn big LDA models from big data. The main features of Peacock include hierarchical distributed architecture, real-time prediction and topic de-duplication. We empirically demonstrate that the Peacock system is capable of providing significant benefits via highly scalable LDA topic models for several industrial applications.