Generative Models as a Complex Systems Science: How can we make sense of large language model behavior?
This addresses the problem of lacking explanations for model behaviors in NLP, which is incremental as it builds on existing benchmarking efforts.
The paper argues that understanding large language model behavior requires decomposing it into categories that explain cross-task performance, to guide mechanistic explanations and future-proof research.
Coaxing out desired behavior from pretrained models, while avoiding undesirable ones, has redefined NLP and is reshaping how we interact with computers. What was once a scientific engineering discipline-in which building blocks are stacked one on top of the other-is arguably already a complex systems science, in which emergent behaviors are sought out to support previously unimagined use cases. Despite the ever increasing number of benchmarks that measure task performance, we lack explanations of what behaviors language models exhibit that allow them to complete these tasks in the first place. We argue for a systematic effort to decompose language model behavior into categories that explain cross-task performance, to guide mechanistic explanations and help future-proof analytic research.