Twitter-Demographer: A Flow-based Tool to Enrich Twitter Data
This tool addresses the need for enriched datasets for NLP practitioners and computational social scientists, but it is incremental as it builds on existing flow-based programming paradigms without introducing new methods.
The paper tackles the problem of insufficient variables in Twitter data for NLP and social science research by introducing Twitter-Demographer, a flow-based tool that enriches tweets with additional information like location and sentiment, aiming to improve reproducibility and privacy.
Twitter data have become essential to Natural Language Processing (NLP) and social science research, driving various scientific discoveries in recent years. However, the textual data alone are often not enough to conduct studies: especially social scientists need more variables to perform their analysis and control for various factors. How we augment this information, such as users' location, age, or tweet sentiment, has ramifications for anonymity and reproducibility, and requires dedicated effort. This paper describes Twitter-Demographer, a simple, flow-based tool to enrich Twitter data with additional information about tweets and users. Twitter-Demographer is aimed at NLP practitioners and (computational) social scientists who want to enrich their datasets with aggregated information, facilitating reproducibility, and providing algorithmic privacy-by-design measures for pseudo-anonymity. We discuss our design choices, inspired by the flow-based programming paradigm, to use black-box components that can easily be chained together and extended. We also analyze the ethical issues related to the use of this tool, and the built-in measures to facilitate pseudo-anonymity.