Mining Half a Billion Topical Experts Across Multiple Social Networks
This addresses the need for scalable expert identification across social media for applications like recommendation systems, but it is incremental as it builds on existing expertise mining methods with a larger dataset.
The study tackled the problem of identifying topical experts across multiple social networks by combining data from Twitter, Facebook, Google+, LinkedIn, Wikipedia, and webpages, analyzing 37 features from over 12 billion messages and 58 billion social graph edges. The result was a production system that ranks over 650 million experts in more than 9,000 topical domains daily, with validation showing features like Twitter Lists and Wikipedia as effective indicators.
Mining topical experts on social media is a problem that has gained significant attention due to its wide-ranging applications. Here we present the first study that combines data from four major social networks -- Twitter, Facebook, Google+ and LinkedIn, along with the Wikipedia graph and internet webpage text and metadata, to rank topical experts across the global population of users. We perform an in-depth analysis of 37 features derived from various data sources such as message text, user lists, webpages, social graphs and wikipedia. This large-scale study includes more than 12 billion messages over a 90-day sliding window and 58 billion social graph edges. Comparison reveals that features derived from Twitter Lists, Wikipedia, internet webpages and Twitter Followers are especially good indicators of expertise. We train an expertise ranking model using these features on a large ground truth dataset containing almost 90,000 labels. This model is applied within a production system that ranks over 650 million experts in more than 9,000 topical domains on a daily basis. We provide results and examples on the effectiveness of our expert ranking system, along with empirical validation. Finally, we make the topical expertise data available through open REST APIs for wider use.