Language Models Understand Us, Poorly
This addresses the philosophical and practical problem of whether language models truly understand language, which is foundational for AI researchers and ethicists, but it is incremental in synthesizing existing views.
The paper investigates three views of human language understanding and argues that internal representations are sufficient for understanding, while reviewing state-of-the-art models as pragmatically challenged and questioning the scaling paradigm's limits.
Some claim language models understand us. Others won't hear it. To clarify, I investigate three views of human language understanding: as-mapping, as-reliability and as-representation. I argue that while behavioral reliability is necessary for understanding, internal representations are sufficient; they climb the right hill. I review state-of-the-art language and multi-modal models: they are pragmatically challenged by under-specification of form. I question the Scaling Paradigm: limits on resources may prohibit scaled-up models from approaching understanding. Last, I describe how as-representation advances a science of understanding. We need work which probes model internals, adds more of human language, and measures what models can learn.