What We Talk About When We Talk About LMs: Implicit Paradigm Shifts and the Ship of Language Models
This work addresses the issue of implicit paradigm shifts in scientific terminology for researchers in NLP and related fields, offering a novel perspective on scientific progress.
The paper tackles the problem of how the term 'Language Models' evolves implicitly in scientific discourse, analogous to the Ship of Theseus, by constructing data infrastructure from NLP publications and performing text-based analyses to quantify its use as a term of art.
The term Language Models (LMs) as a time-specific collection of models of interest is constantly reinvented, with its referents updated much like the $\textit{Ship of Theseus}$ replaces its parts but remains the same ship in essence. In this paper, we investigate this $\textit{Ship of Language Models}$ problem, wherein scientific evolution takes the form of continuous, implicit retrofits of key existing terms. We seek to initiate a novel perspective of scientific progress, in addition to the more well-studied emergence of new terms. To this end, we construct the data infrastructure based on recent NLP publications. Then, we perform a series of text-based analyses toward a detailed, quantitative understanding of the use of Language Models as a term of art. Our work highlights how systems and theories influence each other in scientific discourse, and we call for attention to the transformation of this Ship that we all are contributing to.