From Static Inference to Dynamic Interaction: Navigating the Landscape of Streaming Large Language Models
This paper provides a foundational overview and taxonomy for researchers and developers working with large language models in real-time, dynamic environments, clarifying a fragmented and ambiguous field.
This paper addresses the limitations of standard LLMs in dynamic, real-time scenarios by defining and categorizing the emerging paradigm of streaming LLMs. It clarifies existing ambiguities by establishing a unified definition based on data flow and dynamic interaction, and proposes a systematic taxonomy of current streaming LLMs.
Standard Large Language Models (LLMs) are predominantly designed for static inference with pre-defined inputs, which limits their applicability in dynamic, real-time scenarios. To address this gap, the streaming LLM paradigm has emerged. However, existing definitions of streaming LLMs remain fragmented, conflating streaming generation, streaming inputs, and interactive streaming architectures, while a systematic taxonomy is still lacking. This paper provides a comprehensive overview and analysis of streaming LLMs. First, we establish a unified definition of streaming LLMs based on data flow and dynamic interaction to clarify existing ambiguities. Building on this definition, we propose a systematic taxonomy of current streaming LLMs and conduct an in-depth discussion on their underlying methodologies. Furthermore, we explore the applications of streaming LLMs in real-world scenarios and outline promising research directions to support ongoing advances in streaming intelligence. We maintain a continuously updated repository of relevant papers at https://github.com/EIT-NLP/Awesome-Streaming-LLMs.