Lithium NLP: A System for Rich Information Extraction from Noisy User Generated Text on Social Media
This work addresses the challenge of processing noisy social media text for applications like entity and sentiment extraction, but it appears incremental as it builds on existing NLP methods without a major paradigm shift.
The paper tackles the problem of extracting rich information from noisy user-generated text on social media by introducing the Lithium NLP system, which achieves performance comparable to or better than state-of-the-art commercial systems.
In this paper, we describe the Lithium Natural Language Processing (NLP) system - a resource-constrained, high- throughput and language-agnostic system for information extraction from noisy user generated text on social media. Lithium NLP extracts a rich set of information including entities, topics, hashtags and sentiment from text. We discuss several real world applications of the system currently incorporated in Lithium products. We also compare our system with existing commercial and academic NLP systems in terms of performance, information extracted and languages supported. We show that Lithium NLP is at par with and in some cases, outperforms state- of-the-art commercial NLP systems.