AIJan 15

State of AI: An Empirical 100 Trillion Token Study with OpenRouter

arXiv:2601.10088v116 citationsh-index: 2
Originality Incremental advance
AI Analysis

Provides empirical insights into how developers and end-users actually engage with LLMs in practice, which can inform model builders, AI developers, and infrastructure providers.

This study analyzed over 100 trillion tokens of real-world LLM interactions via OpenRouter to understand practical usage patterns, finding substantial adoption of open-weight models, popularity of creative roleplay and coding assistance, and a 'Glass Slipper' effect where early users show longer engagement than later cohorts.

The past year has marked a turning point in the evolution and real-world use of large language models (LLMs). With the release of the first widely adopted reasoning model, o1, on December 5th, 2024, the field shifted from single-pass pattern generation to multi-step deliberation inference, accelerating deployment, experimentation, and new classes of applications. As this shift unfolded at a rapid pace, our empirical understanding of how these models have actually been used in practice has lagged behind. In this work, we leverage the OpenRouter platform, which is an AI inference provider across a wide variety of LLMs, to analyze over 100 trillion tokens of real-world LLM interactions across tasks, geographies, and time. In our empirical study, we observe substantial adoption of open-weight models, the outsized popularity of creative roleplay (beyond just the productivity tasks many assume dominate) and coding assistance categories, plus the rise of agentic inference. Furthermore, our retention analysis identifies foundational cohorts: early users whose engagement persists far longer than later cohorts. We term this phenomenon the Cinderella "Glass Slipper" effect. These findings underscore that the way developers and end-users engage with LLMs "in the wild" is complex and multifaceted. We discuss implications for model builders, AI developers, and infrastructure providers, and outline how a data-driven understanding of usage can inform better design and deployment of LLM systems.

Foundations

The foundational work for this paper's niche, ranked by how specifically the neighbourhood builds on it — not by global fame.

Your Notes