Rare Event Analysis of Large Language Models
This addresses the challenge of unobserved rare events in LLM deployment for developers and users, though it appears incremental as it builds on existing probabilistic modeling concepts.
The paper tackles the problem of rare but significant behaviors in large language models during inference, presenting an end-to-end framework for systematic analysis with practical implementation including generation strategies, probability estimation, and error analysis.
Being probabilistic models, during inference large language models (LLMs) display rare events: behaviour that is far from typical but highly significant. By definition all rare events are hard to see, but the enormous scale of LLM usage means that events completely unobserved during development are likely to become prominent in deployment. Here we present an end-to-end framework for the systematic analysis of rare events in LLMs. We provide a practical implementation spanning theory, efficient generation strategies, probability estimation and error analysis, which we illustrate with concrete examples. We outline extensions and applications to other models and contexts, highlighting the generality of the concepts and techniques presented here.