LLM Cognitive Judgements Differ From Human
This work addresses the problem of understanding AI cognitive alignment for researchers and developers, but it is incremental as it builds on existing cognitive science methods.
The study examined GPT-3 and ChatGPT on a limited-data inductive reasoning task from cognitive science, finding that their cognitive judgements differ from human-like patterns.
Large Language Models (LLMs) have lately been on the spotlight of researchers, businesses, and consumers alike. While the linguistic capabilities of such models have been studied extensively, there is growing interest in investigating them as cognitive subjects. In the present work I examine GPT-3 and ChatGPT capabilities on an limited-data inductive reasoning task from the cognitive science literature. The results suggest that these models' cognitive judgements are not human-like.