AnthroScore: A Computational Linguistic Measure of Anthropomorphism
This addresses concerns about misleading anthropomorphism in computer science discourse and scientific misinformation in mass media, providing a tool for analysis across text sources.
The authors tackled the problem of measuring implicit anthropomorphism in language by developing AnthroScore, an automatic metric using a masked language model, and found that anthropomorphism has increased over 15 years in research papers, especially in language model-related work, and is higher in news headlines than in cited papers.
Anthropomorphism, or the attribution of human-like characteristics to non-human entities, has shaped conversations about the impacts and possibilities of technology. We present AnthroScore, an automatic metric of implicit anthropomorphism in language. We use a masked language model to quantify how non-human entities are implicitly framed as human by the surrounding context. We show that AnthroScore corresponds with human judgments of anthropomorphism and dimensions of anthropomorphism described in social science literature. Motivated by concerns of misleading anthropomorphism in computer science discourse, we use AnthroScore to analyze 15 years of research papers and downstream news articles. In research papers, we find that anthropomorphism has steadily increased over time, and that papers related to language models have the most anthropomorphism. Within ACL papers, temporal increases in anthropomorphism are correlated with key neural advancements. Building upon concerns of scientific misinformation in mass media, we identify higher levels of anthropomorphism in news headlines compared to the research papers they cite. Since AnthroScore is lexicon-free, it can be directly applied to a wide range of text sources.