Position: Key Claims in LLM Research Have a Long Tail of Footnotes
This work addresses foundational issues in ML/AI research by highlighting and scrutinizing vague claims that affect the entire field, making it incremental in its critical re-examination.
The paper tackles the problem of undefined terminology and unexamined assumptions in LLM research by providing a definition of LLMs and critically analyzing five common claims about their properties, such as emergent properties, to suggest future research directions.
Much of the recent discourse within the ML community has been centered around Large Language Models (LLMs), their functionality and potential -- yet not only do we not have a working definition of LLMs, but much of this discourse relies on claims and assumptions that are worth re-examining. We contribute a definition of LLMs, critically examine five common claims regarding their properties (including 'emergent properties'), and conclude with suggestions for future research directions and their framing.