Taming the Centaur(s) with LAPITHS: a framework for a theoretically grounded interpretation of AI performances
For AI researchers, this work provides a principled method to counteract behavioristic interpretations of language model performances as evidence of human-like cognition.
The paper introduces LAPITHS, a framework to evaluate claims of human-like cognition in AI models, and demonstrates that CENTAUR's claims of being a unified model of cognition are not justified, showing that similar results can be reproduced by systems lacking cognitive plausibility.
We introduce a framework called LAPITHS (Language model Analysis through Paradigm grounded Interpretations of Theses about Human likenesS) and use it to show that several major claims advanced by models such as CENTAUR, proposed as an artificial Unified Model of Cognition, are not theoretically or empirically justified. LAPITHS provides a principled reference point for counteracting the current behaviouristic tendency in AI research to interpret the human level performances of transformer based language models as evidence of human like underlying computation and, by extension, as signs of cognitive abilities. The novelty of LAPITHS lies in making explicit the arguments grounded in two quantitative assessments: (i) the Minimal Cognitive Grid, a theoretically motivated method for estimating the cognitive plausibility of artificial systems, and (ii) a behavioural comparison showing that results similar to those reported for CENTAUR like models can be reproduced by other systems that do not satisfy the structural constraints typically associated with cognitive plausibility, and whose outputs do not provide independent explanatory insight into human cognition.