Moran Cerf

Heinrich Peters, Moran Cerf, Sandra C. Matz

This study investigates the capacity of Large Language Models (LLMs) to infer the Big Five personality traits from free-form user interactions. The results demonstrate that a chatbot powered by GPT-4 can infer personality with moderate accuracy, outperforming previous approaches drawing inferences from static text content. The accuracy of inferences varied across different conversational settings. Performance was highest when the chatbot was prompted to elicit personality-relevant information from users (mean r=.443, range=[.245, .640]), followed by a condition placing greater emphasis on naturalistic interaction (mean r=.218, range=[.066, .373]). Notably, the direct focus on personality assessment did not result in a less positive user experience, with participants reporting the interactions to be equally natural, pleasant, engaging, and humanlike across both conditions. A chatbot mimicking ChatGPT's default behavior of acting as a helpful assistant led to markedly inferior personality inferences and lower user experience ratings but still captured psychologically meaningful information for some of the personality traits (mean r=.117, range=[-.004, .209]). Preliminary analyses suggest that the accuracy of personality inferences varies only marginally across different socio-demographic subgroups. Our results highlight the potential of LLMs for psychological profiling based on conversational interactions. We discuss practical implications and ethical challenges associated with these findings.

Sandra C. Matz, Heinrich Peters, Moran Cerf et al.

As artificial intelligence (AI) models become an integral part of everyday life, our interactions with them shift from purely functional exchanges to more relational experiences. For these experiences to be successful, artificial agents need to be able to detect and interpret social cues and interpersonal dynamics; both within and outside of their own human-agent relationships. In this paper, we explore whether AI models can accurately decode one of the arguably most important but complex social signals: romantic attraction. Specifically, we test whether Large Language Models can detect romantic attraction during brief getting-to-know-you interactions between humans. Examining data from 964 speed dates, we show that ChatGPT can predict both objective and subjective indicators of speed dating success (r=0.12-0.23). Although predictive performance remains relatively low, ChatGPT's predictions of actual matching (i.e., the exchange of contact information) were not only on par with those of human judges but incremental to speed daters' own predictions. In addition, ChatGPT's judgments showed substantial overlap with those made by human observers (r=0.21-0.35), highlighting similarities in their representation of romantic attraction that are independent of accuracy. Our findings also offer insights into how ChatGPT arrives at its predictions and the mistakes it makes. Specifically, we use a Brunswik lens approach to identify the linguistic and conversational cues utilized by ChatGPT (and human judges) vis-a-vis those that are predictive of actual matching.