The unreasonable effectiveness of pattern matching
This addresses debates about the nature of LLMs, showing pattern-matching as a core component of intelligence, though it is incremental in understanding model capabilities.
The study found that large language models can accurately translate sentences with nonsense words into meaningful English, demonstrating their ability to infer meaning from structural patterns alone.
We report on an astonishing ability of large language models (LLMs) to make sense of "Jabberwocky" language in which most or all content words have been randomly replaced by nonsense strings, e.g., translating "He dwushed a ghanc zawk" to "He dragged a spare chair". This result addresses ongoing controversies regarding how to best think of what LLMs are doing: are they a language mimic, a database, a blurry version of the Web? The ability of LLMs to recover meaning from structural patterns speaks to the unreasonable effectiveness of pattern-matching. Pattern-matching is not an alternative to "real" intelligence, but rather a key ingredient.