Theoretical foundations and limits of word embeddings: what types of meaning can they capture?
This work addresses the theoretical foundations of word embeddings for cultural sociologists, offering insights into their capabilities and limitations in capturing meaning, but it is incremental in building on existing linguistic critiques.
The paper examines how word embeddings model three core premises of structural linguistic theory—relational, coherent, and static meaning—and identifies both vulnerabilities to critiques and novel solutions. It argues that formalizing meaning with embeddings can clarify concepts in cultural sociology, such as meaning coherence, by pushing for specification and reimagination.
Measuring meaning is a central problem in cultural sociology and word embeddings may offer powerful new tools to do so. But like any tool, they build on and exert theoretical assumptions. In this paper I theorize the ways in which word embeddings model three core premises of a structural linguistic theory of meaning: that meaning is relational, coherent, and may be analyzed as a static system. In certain ways, word embedding methods are vulnerable to the same, enduring critiques of these premises. In other ways, they offer novel solutions to these critiques. More broadly, formalizing the study of meaning with word embeddings offers theoretical opportunities to clarify core concepts and debates in cultural sociology, such as the coherence of meaning. Just as network analysis specified the once vague notion of social relations (Borgatti et al. 2009), formalizing meaning with embedding methods can push us to specify and reimagine meaning itself.