A Philosophical Introduction to Language Models - Part II: The Way Forward
It addresses philosophical issues for researchers and theorists in AI and cognitive science, but is incremental as it builds on existing debates without introducing new methods or data.
The paper explores philosophical questions raised by large language models, focusing on interpretability, consciousness debates, and implications for modeling human cognition, without presenting new empirical results or concrete numbers.
In this paper, the second of two companion pieces, we explore novel philosophical questions raised by recent progress in large language models (LLMs) that go beyond the classical debates covered in the first part. We focus particularly on issues related to interpretability, examining evidence from causal intervention methods about the nature of LLMs' internal representations and computations. We also discuss the implications of multimodal and modular extensions of LLMs, recent debates about whether such systems may meet minimal criteria for consciousness, and concerns about secrecy and reproducibility in LLM research. Finally, we discuss whether LLM-like systems may be relevant to modeling aspects of human cognition, if their architectural characteristics and learning scenario are adequately constrained.