On the Stability of Online Language Features: How Much Text do you Need to know a Person?
This work addresses the reliability of trait inference from online text for researchers and practitioners, but it is incremental as it builds on existing methods with new data.
The study investigated the stability of linguistic features and inferred traits like personality across different online media, finding that models need tuning per medium and providing guidelines for minimum text required for representative results.
In recent years, numerous studies have inferred personality and other traits from people's online writing. While these studies are encouraging, more information is needed in order to use these techniques with confidence. How do linguistic features vary across different online media, and how much text is required to have a representative sample for a person? In this paper, we examine several large sets of online, user-generated text, drawn from Twitter, email, blogs, and online discussion forums. We examine and compare population-wide results for the linguistic measure LIWC, and the inferred traits of Big5 Personality and Basic Human Values. We also empirically measure the stability of these traits across different sized samples for each individual. Our results highlight the importance of tuning models to each online medium, and include guidelines for the minimum amount of text required for a representative result.