Limits for Learning with Language Models
This work identifies foundational limits in LLMs for tasks requiring formal semantic guarantees, which is significant for researchers and practitioners in NLP and AI, though it is incremental in building on prior empirical findings.
The paper tackles the problem of whether large language models (LLMs) can capture fundamental semantic properties, proving theoretically that LLMs cannot learn certain aspects like semantic entailment and consistency, limiting their ability to handle tasks requiring deep linguistic understanding.
With the advent of large language models (LLMs), the trend in NLP has been to train LLMs on vast amounts of data to solve diverse language understanding and generation tasks. The list of LLM successes is long and varied. Nevertheless, several recent papers provide empirical evidence that LLMs fail to capture important aspects of linguistic meaning. Focusing on universal quantification, we provide a theoretical foundation for these empirical findings by proving that LLMs cannot learn certain fundamental semantic properties including semantic entailment and consistency as they are defined in formal semantics. More generally, we show that LLMs are unable to learn concepts beyond the first level of the Borel Hierarchy, which imposes severe limits on the ability of LMs, both large and small, to capture many aspects of linguistic meaning. This means that LLMs will continue to operate without formal guarantees on tasks that require entailments and deep linguistic understanding.