Do GPT Language Models Suffer From Split Personality Disorder? The Advent Of Substrate-Free Psychometrics
This addresses safety concerns for AI systems integrated into human life, though it is incremental in highlighting inconsistencies in existing research.
The study investigated whether GPT language models develop a consistent personality by administering personality questionnaires in nine languages, finding interlingual and intralingual instabilities that indicate a lack of a core personality, which could lead to unsafe AI behavior.
Previous research on emergence in large language models shows these display apparent human-like abilities and psychological latent traits. However, results are partly contradicting in expression and magnitude of these latent traits, yet agree on the worrisome tendencies to score high on the Dark Triad of narcissism, psychopathy, and Machiavellianism, which, together with a track record of derailments, demands more rigorous research on safety of these models. We provided a state of the art language model with the same personality questionnaire in nine languages, and performed Bayesian analysis of Gaussian Mixture Model, finding evidence for a deeper-rooted issue. Our results suggest both interlingual and intralingual instabilities, which indicate that current language models do not develop a consistent core personality. This can lead to unsafe behaviour of artificial intelligence systems that are based on these foundation models, and are increasingly integrated in human life. We subsequently discuss the shortcomings of modern psychometrics, abstract it, and provide a framework for its species-neutral, substrate-free formulation.