What's happening in your neighborhood? A Weakly Supervised Approach to Detect Local News
This work addresses the need for accurate local news detection to improve recommendations for users, local businesses, and neighborhood safety, though it is incremental as it builds on existing NLP methods.
The paper tackles the problem of detecting local news articles by developing a weakly supervised pipeline that incorporates domain knowledge and auto data processing, achieving higher precision and recall compared to the Stanford CoreNLP NER model on a real-world dataset.
Local news articles are a subset of news that impact users in a geographical area, such as a city, county, or state. Detecting local news (Step 1) and subsequently deciding its geographical location as well as radius of impact (Step 2) are two important steps towards accurate local news recommendation. Naive rule-based methods, such as detecting city names from the news title, tend to give erroneous results due to lack of understanding of the news content. Empowered by the latest development in natural language processing, we develop an integrated pipeline that enables automatic local news detection and content-based local news recommendations. In this paper, we focus on Step 1 of the pipeline, which highlights: (1) a weakly supervised framework incorporated with domain knowledge and auto data processing, and (2) scalability to multi-lingual settings. Compared with Stanford CoreNLP NER model, our pipeline has higher precision and recall evaluated on a real-world and human-labeled dataset. This pipeline has potential to more precise local news to users, helps local businesses get more exposure, and gives people more information about their neighborhood safety.