Social Intelligence Data Infrastructure: Structuring the Present and Navigating the Future
This work addresses the problem of fragmented social intelligence data for NLP researchers, providing a structured infrastructure to guide future dataset development, though it is incremental as it organizes existing resources rather than introducing new methods.
The authors tackled the lack of a cohesive framework for social intelligence in NLP by building a Social AI Data Infrastructure with a taxonomy and library of 480 datasets, enabling analysis of current data and identification of future needs like multifaceted datasets and increased diversity.
As Natural Language Processing (NLP) systems become increasingly integrated into human social life, these technologies will need to increasingly rely on social intelligence. Although there are many valuable datasets that benchmark isolated dimensions of social intelligence, there does not yet exist any body of work to join these threads into a cohesive subfield in which researchers can quickly identify research gaps and future directions. Towards this goal, we build a Social AI Data Infrastructure, which consists of a comprehensive social AI taxonomy and a data library of 480 NLP datasets. Our infrastructure allows us to analyze existing dataset efforts, and also evaluate language models' performance in different social intelligence aspects. Our analyses demonstrate its utility in enabling a thorough understanding of current data landscape and providing a holistic perspective on potential directions for future dataset development. We show there is a need for multifaceted datasets, increased diversity in language and culture, more long-tailed social situations, and more interactive data in future social intelligence data efforts.